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Iowa conducted its own experiment on the evaluation of freshman composition. This 
experiment was designed to determine (1) which method of rating papers is more 
reliable — the "general impression" method of the Educational Testing Service or the 
"analyticaf method used at the University of Iowa, (2) if a more significant measure o 
writing quality is obtained when the better of two papers, rather than merely one. is 
used as a criterion, and (3) if the writing performance of students completing the 
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of students were selected are described in detail, as are the methods of 
themes. Chapters discussing findings, conclusions, and recommendations indicate that 
the results of the experiment were inconclusive because the rating of the themes was 
not reliable. Included in the report are extensive tables of data and appendixes listing 
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PROBLEM 



Several years ago the University of Northern Iowa initiated a 
project to test the effectiveness of college-level instruction in 
frCShniEn COrnDOSitlOn- 1“h© #aYne*i^*iTnanf 

from the freshman composition course at each of five institutions 



during the academic year 1964-65 some 325 entering freshmen matched 
with other entering freshmen taking the course. The UNI experiment 
intended to reach some generalizations about the effectiveness of 
freshman composition courses in general. It sought to do this by 
conducting the experiment at five different campuses, each of which 
had a somewhat different type of freshman composition course. The 
University of Iowa was one of the institutions cooperating with the 
University of Northern Iowa in this large project. (At the U of I, 
freshman composition has been taught, along with public speaking, 
in "integrated courses" offered by the Rhetoric Program.) 



To test the effectiveness of the experiment, the students in the 
Experimental Group (those excused from the course) and the students 
in the Control Group (those taking the course as usual) were to be 
examined on four occasions: September, 1964; January, 1965; June, 

1965; and June, 1966. On all four occasions, the testing included 
a two-hour theme examination and two one-hour objective English, 
tests, the College Entrance Examination Board test and the Coopera- 
tive Test of English Expression. The themes were to be evaluated 
at UNI after each testing occasion by raters brought from various 
educational institutions for that purpose and directed by Fred Godshalk 
of the Educational Testing Service. For a detailed account of the 
pilot study whijh preceded the five-institutuion project, see the 
Interim Report . 



Several considerations prompted the University of Iowa to 
undertake a small experiment to supplement the larger project directed 
at U'NI: 



1. As has been shown by a number of investigators,^ 
the day-to-day writing performance of college freshmen 
varies, especially the performance of better writers. 
Consequently, if a large number of the better writers 
happened to perform in a mediocre fashion on the post- 
test themes, they would tend to offset the theme results 



Ross M. Jewell, John Cowley, and Gordon Rhum, The Effectiveness 
of Col lege- Level Instruction in Freshman Composi tion :~Tnterim Report, 

USOE Project No. 2188 (Cedar Falls: University of Northern lowiT^ 

December, 1966) . 

2 Richard Braddock, Richard Lloyd-Jones, and Lowell Schoer, Research 
in Written Composition (Champaign, Illinois: National Council of Teachers 

dlF English , 1563) , pp . 6-7. 
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of the better writers who performed well. There seemed 
to be a serious danger, then, with only one theme being 
written on each of the four testing occasions, that the 
UNI experiment would yield no significant differences, 
like so many experiments of this type. 

2. Because the ETS raters would know which papers 
were pretest themes and which were posttest themes, their 
judgments of the quality of those papers might be affected 
by that knowledge and hence should not be used to determine 
whether or not the freshman composition instruction yielded 
significant improvement in the writing of the students. 

Even though the rating could be used to determine the extent 
to which the Experimental Group or Control Group was superior 
to the other on each testing occasion, one would not be able 
to say confidently that the difference was due to the improve- 
ment of one group rather than to the deterioration of the 
other group. 

3. Under the ETS system, each rater judges 20 to 30 
themes an hour, sometime more. One may hypothesize that, 
proceeding at such a speed, a rater may tend unconsciously 
to base his judgments largely on style of writing and 
mechanics, letting a quick impression of those factors 
color his judgment of the writer’s basic analysis, organi- 
zation and substance. At the U of I, although Rhetoric 
instructors typically mark matters of style and mechanics 

on themes, they usually spend little class time in instruction 
on those matters, placing more emphasis on the analysis, 
organization, and support of ideas. Furthermore, while 
the ETS raters rapidly read a paper and assign it a number 
from one through nine as a general impression of its merit, 
in its departmental writing examinations the Rhetoric Program 
has used an analytical method of grading: Each paper is 

graded on each of five categories, four of them from one 
to five points, the fifth from one to three points. The 
points on the five categories are then added, yielding a 
total score which may range from five to twenty-three 
points. Typically, U of I instructors rate from six to 
eight themes an hour using this analytical system. In 
brief, there was concern that the ETS rating system would 
not evaluate well the matters emphasized in the Rhetoric 
Program at the U of I. 

The small U of I project has sought to supplement the large UNI 
experiment by having the students in the Experimental Group and those 
in the Control Group at the University of Iowa write a second theme 
for the pretest (September, 1964) and for the most demanding of the 
posttest situations (June, 1966, at least a year after the date that 
the Control Group student had completed his work in the freshman 
Rhetoric Program). Then the better of each student’s pretest themes 
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and the better of his posttest themes would be utilized in judging 
the effectiveness of the composition component of the freshman 
course. Furthermore, by having U of I raters judge not only the 
extra pretest and post test themes but also the regular themes 
written on those two occasions, it would be possible to contrast 
the effectiveness of the general impression and analytical methods 
of grading, inasmuch as some of the papers would be judged under 
both systems. Finally, by saving both pretest and both posttest 
papers until the conclusion of the experiment, the U of I raters 
could rate all four papers at the same time, not knowing which 
were pretest and which posttest themes and hence permitting a 
valid determination of the improvement of writing between the 
two tests. 

Briefly, then, this study has sought to supplement the larger 
UNI experiment by seeking answers to these questions: 

1. At the college freshman -sophomore level of 
writing, which is a more reliable method of rating 
papers— the U of I "analytical” method or the ETS 
"general impression" method? 

2. At the college freshman- sophomore level, can 
a more significant measure of writing be obtained when 
the criterion used is the better of two papers, rather 
than merely one paper? 

3. Has the writing performance of students who 
have taken Rhetoric at the University of Iowa improved 
significantly more than that of matched students who 
have been excused from the course? 
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II 

PROCEDURE 



A description of the 
nature of the experiment, 
was in 1964-65. 



Rlietoric Program will clarify the 

^ 1 1 V% ^ ^ c* •%» 4 V\ fX c 

me rrugrani wxx x i/w vtv><9v.x 



1 1 * 



All entering freshmen were required to register for Rhetoric. 

On the basis of his scores on the American College Test, the 
student was assigned either to 10:3, an accelerated one-semester 
course, or to 10:1 and subsequently to 10:2, the regular two- 
semester course. No matter which course the student took, he 
received instruction in written composition, in public speaking, 
and, often in a much less deliberate manner, in the reading of 
nonfictive prose. In each of those courses, the instructor was 
expected to assign at least eleven themes (400-500 words) , eight 
speeches (about four minutes) , and some ten reading assignments 
for which there was some written follow-up. (The departmental 
theme and speech examinations were usually counted toward those 
minima.) The 10:1 course emphasized exposition in writing, 
speaking, and reading: formulating a central idea, organizing 

discourse, and supporting generalizations with facts, examples, 
and other material AltLugh the instructor might spend some 
time in class on sentence style and diction, he was e>.pected to 
treat the more obvious matters of grammar, usage, and mechanics 
only as he marked individual papers. In 10:2 the emphasis was 
on critical thinking, the writing of a research paper, and argu- 
mentative writing and speaking. The instructors of the accelerated 
course, 10:3, usually spent two or three weeks on expository writing 
and speaking, and then emphasized the same considerations that they 
would in 10:2. The courses were taught largely by part-time graduate 
assistants pursuing advanced degrees either in the Department of 
English (about 60 per cent) or the Department of Speech and Dr^atic 
Art (about 40 per cent). About two-thirds of the graduate assistants 
were working on the Ph.D. degree and had taught before that year, ^ 
either in the Rhetoric Program or at some other college or university. 

During their initial registration period when they first arrived 
on campus, the students assigned to 10:3 were given an opportunity to 
test out of the Rhetoric requirement by taking a two-hour theme exam 
and a four-minute extemporaneous speaking exam. If a student did at 
least B- work on each of these exams, he was excused from the Rhetoric 
requirement— and hence from the experiment. If he did well on the 
speech exam only, he could transfer, from 10i3 to 10:4, a two-hour 
course in the writing aspect of 10:3. If he did well on the writing 
exam but not on the speech exam, he transferred from 10.3 to 36.25, 
a public- speaking course offered by the Department of Speech and 
Dramatic Art. Students transferred to 10:4 were kept in the experi- 
ment; those transferred to 36:25, of course, were not. 
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S elect ion and Motivation of Students 

In April, 1964, the selection of students for the Experimental 
Group took place while the students were still in high school. The 
sampling procedures established by UNI for the larger experiment 
were followed in detail at the U of I. Although these procedures 
undoubtedly will be described in detail in the forthcoming final 
report of the larger experiment, a brief description is made here. 
Working with the Tentative Admission Statements of 1,748 prospec- 
tive freshmen for whom ACT scores were already available, a tabu- 
lation was made of the total number of males and of females achieving 
each standard score on the English composition subtest of the ACT 
battery. Then the proportion of males and of females at each of 
those levels was selected, using a table of random, numbers, that 
was necessary to make up a total of 331 students for the Experimental 
Group. A similar procedure was then employed to select some 770 
students to constitute a Control Group pool from which UNI was later 
to select students to match with individuals in the Experimental 
Group. (By the time two years had gone by, only 79 matched pairs 
of students had completed the smaller experiment by having written 
all four themes and answered a brief questionnaire.) 

That July, letters were written to the students in each group 
apprising them of the experiment and their role in it. (Copies of 
these and other letters mentioned below are included in Appendix A.) 
On each letter was typed the room and building where the student was 
to report to take the tests in September. Care was taken to assign 
equal proportions of Experimental and Control students to each of 
the testing rooms in an attempt to equalize among the Experimental 
and Control students any distracting elements or other matters 
associated with particular testing situations which might tend to 
bias the results. The same practice was followed in assigning 
students to rooms for all of the later tests. 

As can be seen from the letters included in Appendix A, a 
careful attempt was made to equalize the motivation of students 
in the Experimental and Control groups. This was done by insisting 
that each student take the test but that no student’s status would 
be affected by the outcome of the test. At the time, this seemed 
to be the only fair and practical way of equalizing motivation-- 
removing all motivation! It did not seem right to have students 
in the Control Group take the experiment exams for their final 
departmental exams in the Rhetoric Program; these students might 
all apply themselves with such diligence that they would surpass 
the Experimental Group largely for that reason. It had not seemed 
fair, either, to have the students in the Experimental Group take 
those exams as departmental final exams after having been excused 
from the Rhetoric requirement for two years; if they did poorly and 
had to enroll for Rhetoric after the midpoint in their undergraduate 
careers, it might be quite difficult for many of them to schedule 
the freshman course as juniors or seniors. 



^ Table XXIII in Helen M. Walker and Joseph Lev, Statistical 
Inference (New York: Holt, Rinehart, and Winston, 1953), pp. 484-485. 
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One must not forget the varied feelings that the participants 
in such an experiment must have. One letter from a student who 
resented taking the examinations in May, 1966, is enclosed at the 
end of Appendix A. Although this is the only such letter received, 
a number of other students revealed much the same feelings in tele- 
phone calls or personal conversations. That is one reason why, when 
a reminder about the last test was mailed on May 20, 1966, the 
students were invited to have coffee and doughnuts before beginning 
the last test. That is also why a questionnaire about their own 
attitudes toward Rhetoric was administered to them during the last 
testing period. (See Appendix B.) 

While reading this report, one must remember that this smaller 
study was imbedded in the larger one directed by UNI. Although the 
students’ attitudes may well have been affected by the fact that they 
wrote a. theme and took two objective tests on each of four different 
occasions, this smaller study concerns only the themes (herein after 
labeled "UNI themes") planuod for the first and fourth semesters in 
the regular experiment and the additional themes (herein after referred 
to as "U of I themes") not planned as part of the regular experiment, 
but also offered on the first and fourth testing occasions. The 
schedule follows for the four themes which form the basis of this 

smaller study: 



Theme 


#1 


(U of I) 


Sept. 15, 1964 


8-10 a.m. 


Theme 


#2 


(UNI) 


Sept. 17, 1964 


7-9 p.m. 


Theme 


#3 


(U of I) 


May 25, 1966 


7-9 p.m. 


Theme 


#4 


(UNI) 


June 3, 1966 


7:30-9:30 a.m. 



The student attitude questionnaire referred to in the previous paragraph 
was given just before the students wrote Theme #3. 

Examination of the instructions for the four themes reveals several 
differences between the and the U of I instructions. (See Appendix C.^ 

Whereas the UNI assignments attempt to put all students on the same 
footing insofar as the topic is concerned by assigning the same topic, 
the U of I instructions assume that an important part of writing is 
the selection of a topic appropriate for the writer and reader, and^ 
the consequent focus of that topic so that it can be treated adequately 
in the time available. Whereas the UNI assignments attempt to help 
the student get started by offering him some basic discussion of the 
topic, the U of I instructions omitted such stimulus material on the 
assumption that the student should provide his own and not be directed 
toward some approach which might tend to minimize differences among 
the student writers. A third difference between the two types of 
theme assignments stemmed from the fact that the UNI assignments were 
concerned only with differences between the Experimental Group and 
Control Group on any one testing occasion, while the U of I assignments 
were also designed to permit comparison of the pretest and posttest 
writing. This difference may not be important inasmuch as the UNI 
pretest and posttest assignments were very close in nature. 



o 
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The analytical method used by the U of I raters to evaluate the 
papers is explained in detail on those pajes after the first in 
Appendix D. The same criteria had been used for some years as the 
basis for evaluating departmental writing examinations in the Rhetoric 



Program and as a basis for instruction of students. Under this system 
separate judgments are made on the quality of each of five aspects of 

o 1 *1 y of* “I on 

lie papvx « Vieiicxc^j. a>«ava o*»cax/ ^ ^ a ^ 

Expression, and Literacy, Each category is rated from one to five 
(F to A) except Literacy, which is rated from one to three CF to C) 
on the theory that no paper should be elevated to an above-average 
position merely because of the absence of literacy errors. To arrive 
at a total score for a paper, one adds the separate scores given the 
five categories. Thus, when one instructor is rating a paper, the 
theoretical range of possible total scores is from five to 23. (In 
the rating for the experiment, however, the total scores of two raters 
were added together, making a theoretical range of from ten to 46. 
Because the ETS raters in the larger experiment used a scale of from 
one to 9 and similarly added the ratings of two judges, the theoretical 
range of total scores under that system was from two to 18.) 



Selection of Raters 



All of the raters were graduate students at the University of 
Iowa who had been teaching at least one year in the Rhetoric Program 
and who seemed to have demonstrated their reliability in judging 
themes. The experience of each of the raters is summarized in 
Appendix E. 

At the beginning of each academic year and towar* the end of 
each semester, the faculty of the Rhetoric Program have meetings 
designed to develop consistency in theme grading. Each instructor 
brings to the meeting his copies of two dittoed themes which he has 
rated beforehand. The Coordinator of the Program discusses his own 
ratings of those two themes, then distributes copies of a third 
theme which the instructors rate independently and submit to the 
Coordinator before they leave. The ratings of these third themes 
at the January 6 and May 9, 1966, meetings provided the basis for 
selecting raters for the experiment. 

At the January meeting, 106 instructors submitted rated themes. 
At the May meeting (at the end of the spring semester, when fewer 
classes are taught), 80 instructors submitted themes. Of these 
numbers, 73 graduate assistants who submitted themes on both 
occasions constituted the pool from whom the most reliable raters 
were selected in the following manner. 

First the total scores were tallied for all of the faculty 
present at each meeting, as is shown in Table 1. 




TABLE 1 



Distribution of Faculty Total Score Ratings 



Total Score 


January 


May 


22 


0 


0 


21 


1 


0 


20 


1 


3 


ly 


n 


9 


18 


9 


0 


17 


9 


6 


16 


5 


6 


15 


12 


6 


14 


6 


10 


13 


24 


15 


12 


17 


15 


11 


12 


10 


10 


2 


3 


9 


5 


9 

4 # 


8 


1 


1 


7 


0 


1 


6 


0 


0 



Total 106 80 



The median score for each theme was 13. (The reader should note 
that such range in ratings is not uncommon throughout the nation. 
When the Rhetoric Program conducts its departmental final theme 
examinations, each theme is read by two raters who have been 
paired to balance tendencies toward severity and leniency.) 

It is often said that many of the new Rhetoric faculty tend 
to grade more severely in practice rating meetings than they do 
ordinarily. Consequently, it seemed appropriate also to compute 
the mean score of the five "regular" members of the Rhetoric 
faculty who had been teaching in the Program more than one year. 



J. Bowers 
R. Braddock 
L. Kelly 
C. Martin 
R. Wachal 

Mean Score 



January 


May 


14 


13 


13 


15 


17 


16 


18 


14 


15 


. 12 


15.4 


14 



Using the median scores of the total groups and the mean scores 
of the regular faculty as ranges, the following ranges in scores for 
each theme were determined as a basis for an initial screening of 
raters . 

January May 

"Best raters" 13-15 13-14 



"Next best" 



12-16 



12-15 
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The ratings were next examined of graduate assistants whose 
total scores fell in the above groups. Their scores on each of the 
five rating categories were compared to the tabulations of the 
Rhetoric faculty as a whole, which are given in Table 2. 



TABLE 2 



Distribution of Faculty 


Category Ratings 


Category 


Score 


January 

Number 


May 

Number 


Central Idea 


5 


3 


0 


and Analysis 


4 


18 


9 




3 


38 


23 




2 


42 


31 




1 


5 


17 



Supporting 


5 


0 


7 


Material 


4 


13 


27 




3 


28 


29 




2 


50 


16 




1 


15 


1 



Organization 5 

4 

3 

2 

1 



2 

26 

55 

24 

1 



0 

8 

21 

46 

5 



Expression 



5 


1 


1 


4 


19 


18 


3 


57 


29 


2 


26 


26 


1 


3 


6 



Literacy 



3 74 56 

2 31 24 

1 1 0 



It seemed best to make the final selection of raters on the 
basis of the proximity of their scores on each category to the scores 
of the faculty as a whole. Consequently, the modal and the next to 
the modal ratings of the faculty on each category were selected as 
the limits for selection of raters. If the next to the modal and 
the second from the modal ratings were close (no further apart 
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than 5 per cent of the total number of instructors rating that theme), 
then all three scores were included in the limits for selection of 
raters. These limits were as follows: 



Seventeen of the "best raters" or "next best raters" had scores 
so close to the above ratings that all or all but one of the scores 
on the five categories of both themes were within the limits. Thirteen 
of the seventeen accepted the invitation to rate themes for the 
experiment. To obtain the desired fourteen raters, one of the "best 
raters" was enlisted, all of whose category scores but two (one on 
each theme) fell within the limits. The ratings on the two themes 
are given in Table 3 on the next page. 



When the raters were paired, an error may have been made which 
may have helped reduce the reliability of the ratings. Because the 
raters had all been very close in their judgment of the January and 
May themes, the chief investigator ignored their ratings and paired 
them according to his amateur impression of their personal tendency 
toward dominance or submissiveness, factors probably important when 
judges reconcile their ratings. As it happened, some of the "higher" 
raters were paired and some of the "lower" raters were paired, as 
Table 4 shows. 



Category 




Central Idea 
and Analysis 



2-3 



2-3 



Support ing 
Material 



2-3 



3-4 



Organization 



2-4 



2-3 



Expression 



2 - 3 



2-3 



Literacy 



2-3 



2-3 
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TABLE 3 



Ratings of All Categories by Each Judge 



Raters 


Practice 

Rating 

Sessions 


1 

Central Idea 
and Analysis 


1 

Support ing 
Material 


Organization 


Expression 


Literacy 


Total 


S. Bush 


Jan. 


2 


2 


3 


3 


3 


13 




May 


2 


3 


2 


2 


3 


12 


E. Fox 


Jan. 


3 


2 


3 


4 


3 


15 




May 


2 


3 


2 


3 


2 


12 


B. Hodge 


Jan. 


3 


2 


3 


4 


3 


15 




May 


2 


4 


2 


4 


2 


14 


N. Holmes 


Jan. 


2 


2 


3 


2 


3 


12 




May 


2 


3 


3 


4 


3 


15 


D. Home 


Jan. 


3 


3 


4 


2 


3 


15 




May 


2 


4 


1 


3 


3 


13 


P. Jensen 


Jan. 


2 


2 


2 


3 


3 


12 




May 


3 


4 


2 


2 


2 


13 


B. Johnson 


Jan. 


3 


3 


4 


3 


3 


16 




May 


1 


4 


2 


3 


3 


13 


M. Kelly 


Jan. 


2 


2 


3 


2 


3 


12 




May 


1 


3 


3 


2 


3 


12 


L. Kramer 


Jan. 


2 


2 


3 


3 


3 


13 




May 


2 


4 


2 


2 


3 


13 


R. Miller 


Jan. 


2 


2 


2 


3 


3 


12 




May 


2 


3 


2 


3 


3 


13 


E. Pixley 


Jan. 


3 


2 


3 


3 


2 


13 




May 


1 


3 


2 


3 


3 


12 


S. Renner 


Jan. 


3 


2 


3 


2 


2 


12 




May 


2 


4 


3 


3 


3 


15 


G. Stewart 


Jan. 


2 


2 


3 


4 


3 


14 




May 


3 


4 


3 


2 


3 


15 


E. Weihe 


Jan. 


3 


2 


3 


4 


3 


15 




May 


3 


3 


3 


3 


3 


15 






TABLE 4 



<« 

Comparison of Total Scores of Pairs of Judges 



Total Scores 
on Themes 



Pair of Raters 


Jan. 


May 


Sums of 
Totals 


Total 
for Pair 


S. Bush 


13 


12 


25 




M. Kelly 


12 


12 


24 


49 


E. Fox 


15 


12 


27 




B. Johnson 


16 


13 


29 


56 


B. Hodge 


15 


14 


29 




D. Home 


15 


13 


28 


57 


N. Holmes 


12 


15 


27 




S. Renner 


12 


15 


27 


54 


P. Jensen 


12 


13 


25 




R. Miller 


12 


13 


25 


50 


E. Pixley 


13 


12 


25 




L. Kramer 


13 


13 


26 


51 


G. Stewart 


14 


15 


29 




E. Weihe 


15 


15 


30 


59 
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Theme Rating 

When the last themes had been written, all eight themes for 
each matched pair of students were brought together. Then both 
themes written by a matched pair of students on the same occasion 
were assigned to the same pair of raters for grading during the 
same rating period. For example, the theme written by the experi- 
mental student on June 3, 1966, and the theme written by his 
matched control student on the same day, were rated by the same 
pair of raters during the same rating ^ '^riod. This was done, of 
course, to minimize differences attribi- ;able to severity and 
leniency and tc freshness and fatigu', of raters. In addition, each 
pair of themes written by matched students on one occasion was 
assigned to a different pair of raters than any fiomes written by 
the same students on another occasion. Thus, the themes written 
by a matched pair of students on May 25, 1966, were graded by a 
different pair of raters than the themes written by the same 
studients on June 3, 1966. The themes the same pair of students 
wrote on September 15, 1964, were graded by a third pair of raters, 
and so on. Tliis was done to prevent the raters from recognizing 
the handwriting and style of a student and hence from guessing 
in the case of the Rhetoric-aesigned themes --whether the theme was 
written in the fall of 1964 or the spring of 1966. 

Once the themes had been distributed according to raters, the 
sheet which explained the assignment and included the student's 
name and the testing date was removed from each theme. The only 
means of identification now left on the theme was the student's 
identification number and, in many cases, a booklet number. It 
was impossible to tell from the student numbers whether the student 
was in the experimental or control group. It was also highly 
unlikely that the raters could tell from the booklet numbers which 
were scattered over a range of several hundred numbers — whether the 
small experiment themes were written in September, 1964, or May, 1966. 

Because the assignment for the September, 1964, large experiment 
theme (Theme #2) was different from the assignment for theJline, 1966, 
large experiment theme (Theme #4), the raters may have been able to 
guess which was the pretest theme and which the posttest theme. 

Even though the paper used for both the pretest and posttest themes 
for the large experiment was all purchased at the same time and 
stored in approximately the same manner, the pretest theme paper 
became slightly darker in color and could have been distinguished 
from the posttest theme paper if the raters had sought differences. 
But it was impossible for the raters to detennine which of these 
themes were written by students in the experimental group, which 
by students in the control group. 

The small experiment theme assignments were designed to measure 
improvement. That is, the topics for the September, 1964, and the 
May, 1966, small experiment themes (Themes #1 and #3) were the same, 
and the students were instructed not to reveal the time at which 
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they were writing. (See Appendix C.) The raters were instructed to 
turn in any themes if they could tell when they were written. (See 
"Instructions to Raters," Appendix B.) Five such themes were turned 
in. All themes of those five students were thereupon withdrawn from 
consideration in the U of I experiment. 

The rating extended from the morning of Wednesday, June 8, 1966, 
to shortly after noon on Saturday, June 11, following this schedule: 



8:30-10 a.m. 
10-10:30 a.m. 
10: 30-noon 
noon-1 p.m. 



Rating 

Coffee Break 
Rating 
Lunch Break 



1-2:30 p.m. Rating 
2:30-3 p.m. Coffee Break 
3-4:30 p.m. Rating 



Hie first period on the first day was spent explaining the nature of 
the experiment, going over the "Instructions to Raters" (Appendix D) , 
and discussing and rating two duplicated themes. These and the pairs 
of themes used for discussion on Thursday and Friday mornings were 
Theme #1 and Theme #2 papers written by students who had dropped out 
of the experiment. Half an hour or more at the beginning of each of 
the three mornings was spent discussing and rating the sample themes . 
(A similar discussion might well have been held each afternoon, too.) 
Otherwise, the chief investigator (who chaired the rating sessions) 
kept aloof from the rating except to distribute and collect papers, 
answer an occasional question about procedure, and see that the group 
observed the time schedule. (Some raters would have worked through 
the breaks but were not permitted to do so.) 

All of the rating sessions met in the same place, Michigan Room 
on the third floor of the Iowa Memorial Union. The room was air- 
conditioned and equipped with more long tables than the raters could 
use. It was removed from active, distracting areas of the Union, 
yet within a few minutes of the first floor and basement cafeterias 
for coffee and lunch breaks. 

The raters were psid $4 an hour ($28 per day) for their services 
and seemed pleased. Rating six hours per day for three and a half 
days, the 14 raters read 672 themes (each of the themes twice, 97 of 
them^four times, five of those six times) at a rate of some five and 
a quarter themes per rater per hour, but that time included not only 
individual silent rating but conferences between judges and daily 
discussion of the same themes by the entire group. 

After the first few rating periods, one-fourth of all the themes 
which had been rated were placed at regular intervals among the themes 
which were still to be rated. The purpose of this was to compare the 
ratings of different pairs of raters on the same themes and thus to 
obtain an indication of the reliability of the rating. In each case, 
a pair of themes written by the same matched pair of students on the 
same occasion was moved together into a new packet of themes being 
scored by a pair of raters other than those who had previously judged 
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the pair of papers. Unfortunately, accomplishing this had not been 
planned adequately beforehand, and an even sampling of the four themes 
was not obtained. The reratings occurred as follows: 



Theme 


N Rerated 
Once 


N Rerated 
Twice 


Total 

Reratings 


1 (U of I) 


6 


0 


6 


2 (UNI) 


38 


1 


40 


3 (U of I) 


25 


0 


25 


4 (UNI) 


28 


4 


36 


Total Reratings 


97 


5 


107 



Though an even distribution of reratings among the four themes 
was not obtained, the identity of the writers and the results of any 
first ratings were successfully hidden from all raters. The raters 
were sometimes conscious of the fact that themes had already been 
rated, but they could not tell who had rated them before or what the 
previous ratings had been. If they themselves had rated the papers 
before, they returned them without rating them a second time. ^Vhen 
all of the rating was completed, each rater filled out a blank 
answering these questions: 



1. Except for any themes which you turned in to 
be disregarded in the experiment for this reason, were 
you able to tell whether any theme was written by an 
experimental or a control group student? 

2. Did you notice any differences between the 
U of I theme booklets written in September, 1964, 
and those written in May, 1966? What differences 
did you detect? 

3. What percentage of the papers you read 
would you estimate were dashed off with little or 
no attempt to be well written? 

All 14 raters answered "No” to each of the first two questions. 

Their answers to the third question varied as follows: 



31-40% 4 

21-30% 3 

11 - 20 % 2 

1-10% 3 

"Some" 1 

"Can't tell" 1 



Though they were not asked anything of this nature, four of the 
raters added notes on their questionnaires, volunteering the 
opinion that there were more "dashed off" papers among those 
written on the "Non-conformity" and "Unconventional" topics 
(the UNI topics) than on the U of I topics, from which students 
had a choice. 
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III 

FINDINGS^ 



This experiment sought to supplement a larger experiment, 

•^he University of Northern Iowa, by seeking answers to 

these questions y 

1. At the college freshman- sophomore level of 
writing, which is a more reliable method of rating 
papers— the U of I ’’analytical” method or the ETS 
’’general impression” method? 

2. At the college freshman- sophomore level, ^ 
can a more significant measure of quality of writing 
be obtained when the criterion used is the better of 
two papers, rather than merely one paper? 

3. Has the writing performance of students who 
have taken Rhetoric at the University of Iowa improved 
significantly more than that of matched students who 
have been excused from the course? 



Analytical versus Impression Rating 

In the larger UNI experiment, two ETS raters judged each of the 
regular themes independently and recorded their scores wit, out 
consulting each other. A rating reliability was computed from 
these pairs of scores, using the Pearson 

With the U of I ratings, a different procedure had to be used becMse 
MCh pair of raters consulted about its ratings and reconciled them 
within certain limits. (See "Instructions to Raters." Appendix D.j 
In the U of I experiment, approximately one-seventh of the themes 
already rated were mixed in with the other themes and rated again 
by other pairs of raters. In no case did the second pair of raters 
have any knowledge of the ratings assigned by the first to 

analysis of the reliability of the rating between pairs of waters 
was made, using the Pearson product-moment forumula. The resulti g 
reliabilities for the U of I and ETS raters are given in Table 5 , 
an examination of which yields the following findings: 



The rating of Central Idea and Analysis by the 
U of I raters was 1 e s s reliable on the UNI themes than 
on the U of I themes. This difference may be attributed to 



1 Acknowledgment is due Gordon Vfhite (University of Iowa) 
for writing the computer program and assembling its analyses. 

^ Professor Jewell and his colleagues helpfully supplied 
the ETS ratings for both themes. 



TABLE .5 



Reliability of U of I and ETS Ratings 
Determined by the Pearson Product -Moment Formula 
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f-H (0 
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Theme 1 

(U of I Pretest 


.48 


-.26 


-.23 


i33 


v64 


-.04 


— 


Theme 2 
(UNI Pretest) 


-.15 


.15 


.14 


-.10 


.74 


.05 


.25 


Theme 3 

(U of I Posttest) 


.40 


.24 


.30 


.42 


.24 


.29 


— 


Theme 4 
(UNI Posttest) 


.13 


.41 


.34 


-.04 


.80 


.34 


.60 


Themes 1+3 


.44 


.19 


.17 


.39 


.36 


.30 


— 


Themes 2+4 


.09 


.35 


.30 


-.07 


.76 


.27 


.46 


Themes 1 +2+3+4 


.22 


.35 


.27 


.02 


.63 


.28 
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the wording of the theme topics. The UNI topics may- 
part ly have focused the student's central idea for 
him, establishing such a common approach from one 
student to the next that the raters had difficulty 



discriminating differences from one paper to another. 

1 lie U OX 1 LUpXk^D Wc?x C U1 WaVlJL/ Ovf 

student would have to focus his central idea more or 



less deliberately or sharply, producing more differences 
to discriminate. With the U of I topics, the student 



may also have had more opportunity to write on something 
interesting to him and, partly because of that and partly 
because of the diversity of topics from which the students 
coul choose, the raters may have tended to find the 
results more or less original, hence easier to discriminate 
among. (Because of the type of analysis used, the variances 
which could be examined to support or refute this hypothesis 
were not printed out while the reliability computations were 
being made.) 



2. Supporting Material was rated less reliably on 
the U of I papers than on the UNI themes. The difference 
seems unaccountable, especially when the suggestion of 
some specific details in the explanation of the UNI topics 
may have tended to reduce the differences among the UNI 
themes in this category. On the other hand, perhaps the 
fact that all the UNI themes were on the same topics 
permitted the raters to concentrate on Supporting Material 
and to notice differences which, on the U of I papers, 
were partly obscured by differences in topic. 



3. Organization was rated less reliably on the 
U of I than on the UI'II themes. Again, the assignment 
of a single topic on each UNI theme may have helped 
the raters distinguish differences in quality of a 
factor like Organization. 



4. Tlie Expression category was rated less reliably 

on the UNI themes than on the U of I papers. No explanation 
is yet apparent, unless giving the student a choice among 
different topics afforded him more opportunity to exploit 
his own expressive powers and hence permitted students to 
achieve a range which tlie raters could discriminate better. 

5. Literacy was rated more reliably than any other 
category. Except for Theme 3, the low figure for which 
seems unaccountable, the reliabilities ranged from ,64 
to .80 in this category. This satisfactory reliability 
is especially noteworthy in the face of the fact that 
Literacy was rated from 1 to but 3, the other four 
categories being rated from 1 to 5. Undoubtedly the 
reliability may be attributed to the point-scoring 
under Literacy used in the U of I system and the more 
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readily apparent correctness and incorrectness of items 
under Literacy than of items under Expression, in which 
point-scoring is also used. (See Appendix D.) 



6. Tlie Total Scores of the pretest themes were 
reached less reliably than the Total Scores of the 
posttest themes. Differences between the U of I and UNI 

trin‘ir*C crornrk r\-P c* o •♦-4 o 

^ W W W A jp VT V* « » V> i W A OWUlXx \/l. X I V' ^ W X V Ci X V/ J 1 0 

made above, and the relative definiteness with which 
Literacy may be rated explains why the reliability of 
its rating surpasses that of the other categories. But 
these differences cancel out each other in a comparison 
of the reliabilities of the ratings of the two pretest 
themes to the reliabilities of the two posttest themes. 

The fact that all four themes were rated at the same 
time undAr the same conditions further obscures an 
explanation. — In any case, the ratings of the pretest 
themes were so unreliable as to make the experiment 
absolutely worthless as a measurement of improvement 
or lack of improvement. 



7. To compare the reliability of the ETS raters 
most carefully to the U of I raters, one must compare 
their ratings on Themes 2 and 4, the only papers which 
both groups of raters judged. The reliability of the 
U of I ratings on both themes taken together was .27, 
compared to .46 for the ETS ratings. Each of these 
figures evidently was reduced by the low reliability of 
ratings on Theme 2 (the pretest theme) — .05 for the 
U of I raters, .25 for the ETS raters. Even on Theme 4, 
however, the U of I ratings were no better than .34 and 
the ETS ratings .60. It is only this last figure which 
approaches a reliability lending any weight to the ratings. 



Better of Two P apers versus One Paper 

One of the major reasons for conducting this small study to 
supplement the larger UNI-directed experiment stemmed from the fear 
that use of only one criterion theme as a posttest would not yield 
significant differences. T}\e rationale is that better writers vary 
more in their day-to-day performance than worse writers and hence 
that some improvement in writing is necessarily obscured by day-to- 
day variation unless writers have several opportunities to demonstrate 
their improvement. Consequently, several analyses were undertaken 
to determine whether or not significant differences could be detected 
using the better of two post test theme scores or the sum of the two 
posttest theme scores (when the UNI design called for only one 
posttest theme at this time — June, a year after the conclusion 
of instruction) . 



20 



3 

Using Lindquist’s Type I design and employing the ANOVA 
computer program, three tests were made of each analysis 
undertaken: 



1. Were there significant differences between 
the mean theme scores of the experimental and control 
groups? 

2. Were there significant differences in mean 
theme scores between levels of ability? 



3. Were there significant interactions of 
treatment with ability? That is, were there differences 
between the experimental and control groups at one level 
even if not at other levels? 



Levels of ability were established by assigning each student to one of 
three levels according to the sum of the two socres awarded him by 
the ETS raters on Theme #2: 



Scores 



No. of Pairs 
Levels of Students- 



13 - 18 L 3 5 

8-12 L 2 60 

2-7 Lj 14 



Analyses were conducted of each of the five categories on the U of I 
theme rating system, of the Total Score, and of the sum of the scores 
on the first three categories. Although all of the findings are given 
in the tables in Appendix F, the major findings are noted here when 
they are significant at or below the .10 level. The reader should 
be warned, however, that when the rating reliability was low, it 
cannot be concluded that the lack of significant differences has 
any significance. 



1 . When the student ’ s better score on Theme 3 or 4 
is the criterion, no significant differences (other tFan^ 
differences by level only) are noted except on the Organi- 
zation category, where the top level of the Control Group 
surpassed the top level of the Experimental Group and 
the bottom level of the Experimental Group surpassed the 
bottom level of the Control Group. The amount of differ- 
ence was small, even though statistically significant. 



^ E. F. Lindquist, Hesign^ An alpis of Expe ri ments 
(Boston: Houghton Mifflin Company, 1?T5>.^V, PP . 
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2. l^en the sum of the student’s two scores on 
Themes 3 and 4 is the criterion, no significant differences 
are noted (other than differences by level only) . 



3. When the student’s gains are computed between the 
better of Theme 1 and 2 and the better of Theme 3 and 4, 

lie? Oiix^ JLXWCtii W UX xo xji \/x^ci.iix4«cxxxQIi^ fviioxv:; cl 

direction is seen which is the reverse of that in Finding 1 
immediately above. With gain scores between better efforts, 
the top level of the Experimental f^roup surpassed the top 
level of the Control Group, and the bottom level of the 
Control Group outperformed the bottom level of the Experimental 
Group. Again, however, the differences, while significant, 
were too small to have practical importance. 



4. When gains are computed on but one pretest and 
one posttest (Themes 1 and 3) , only one significant 
difference is noted, and that is in Expression, favoring 
the Experimental Group over the Control Group by an 
amount too small to make any practical difference. 



Rheto ric Students versus Excused Students 

It has already been noted above that there were very few 
statistically significant differences observed between the Experimental 
and Control Groups and that those differences were small and of no 
practical significance. The absence of observed differences in the 
writing may have come from a failure of the Rhetoric course to have 
an effect or from the failure of the rating to observe an effect 
which did exist. 

In addition to observable differences in the writing itself, 
differences were sought in the attitudes of the students. At the 
May 25, 1966, testing session, the students answered the question- 
naire shown in Appendix B. The results are indicated here in 
Tables 6 and 7. The first question (Table 6) was an attempt to 
determine, unfortunately after the fact, how equivalent were the 
initial attitudes of the Experimental and Control students. They 
seemed roughly equivalent, though there was a tendency for the 
bottom level of the Control Group to appraise its preparation lower 
than the bottom level of the Experimental Group appraised its 
preparation. Doubtless the Control students, having had Rhetoric, 
had developed a clearer picture of their writing deficiencies 
than the Experimental students had been able to develop. 

The second question (Table 7) constituted an attempt to 
determine the students’ inclination to elect Rhetoric if it were 
not required, and hence to derive some impression of whether or 
not the students thought the course was helping them improve 
their writing.. If both the Experimental and Control Groups had 
strongly tended to select answers from 4-6, their positive feelings 
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about the required freshman course would have been unquestioned. 

That was not the case. Only one-fourth of the Experimental 
students indicated answers from 4-6, and the heaviest preference 
(almost half of all the Experimental students) came for an upper- 
classman elective in writing to replace the freshman Rlietoric 
requirement.^ The responses of the Control students were somewhat 

’F’F0T0Tlt . AlmOQt’ n-P f-hoin oawo -racrinncAc 9neuT«k*v»c 4—^ 

— V * “•» ^ ^ -O' ^ w » -w A V V W W ^ ^ f ^ f 

indicating a much stronger endorsement of the course from these 
students who had had it. Did this stronger endorsement stem 
from a recognition of the value of the Rhetoric course or from 
a sense of security in what the students had already experienced? 

Hie answer seems purely speculative. But one piece of speculation 
may go unchallenged by anyone interested in the quality of writing 
at the University of Iowa: when 58 per cent of all the Experimental 

and Control students state after two years on campus that they 
originally entered the University ’’fully" or "rather well” prepared 
to do the writing they had to do in courses other than Rhetoric, 
it is apparent that these other courses do not demand precise, 
incisive writing. 



4 The Experimental students here reject an upperclassman course 
in writing and speaking for one in writing alone. Yet it has been 
demonstrated that the Rhetoric course instruction in speech has a 
statistically significant effect. See Appendix G, "Effectiveness 
of Speech Instruction in the Rhetoric Program." 
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CONCLUSIONS AND RECOMMENDATIONS 



The Cooperative Research Program of the U. S. Office of 
Education invested $4,279 in this project. Unfortunately, the 



•T*l» ^ 

lilt;; \/ I. 



answers sought by the project were not obtained • 
the experiment were inconclusive, most importantly, because the 
rating of the papers was not reliable. Several explanations for 
the unreliability seem apparent: 



1 . The two themes used as a basis for selecting 
raters were both ”C” papers. In addition to ”C" papers, 
clear examples of "A" and "F’’ papers should have been 
used. It may be that the fourteen raters tended to 
keep their ratings near a ’’safe” average range and 
hence not to discriminate differences among papers. 
(Because of the type of analysis used, the variances 
which could be examined to support or refute this 
hypothesis were not printed out while the reliability 
congDutations were being made.) 

2. The rereading of papers to determine rating 
reliability should have been more carefully planned. 
Specifically, a larger proportion of the papers should 
have been reread. (Note that the reliability of the 
ETS ratings was computed on the double rating of all 
papers.) Certainly the sampling of papers to be reread 
should have been carefully selected, not chosen hap- 
hazardly and hence been unrepresentative. 

3. IVhen two or more pretest or two or more 
posttest themes are used (to yield either a best score 
or a total score), probably the same type of assignment 
should be employed for each theme. 

4. More time should have been devoted during 

the rating period to the rating and discussion of themes 
carefully selected to exemplify certain kinds of problems. 
This practice should have been used more extensively to 
build consensus among the raters but, in addi^’^^n, it 
should have been used as a; continual check ag. -st the 
reliability of the raters. If some raters w<- . found 
to be ’’wild” or "too safe” on several such commonly 
graded themes, those raters should have bojn discharged 
from rating any additional papers. 

5. It has been suggested to the authors of this 
report that it might have been better to iise a rating 
scale ranging more widely than the scale normally 
used for instructional purposes. The argument for 
increasing the range is that it will make reliability 



more possible by forcing the raters to make larger 
discriminations among papers. The argument against 
extending the range is that it may tend to reveal small 
amounts of differences which, in the practical world, are 



UJl JLXCI.XC • 



Note that the ratings under the 



U of I system were less reliable than they were under 
the ETS system, even though the range on the U of I scale 
(5-23) is twice as great as it is on the ETS scale (1-9). 
Note further that the best reliabilities shown on Table 5 
were for Literacy, which was graded on a 3-point scale 
in contrast to the 5-noint scale used for the other 
categories . 



6. One explanation for the low reliability pf the 
rating may have been that the differences among the 
papers were in fact very small. If that were true, it 
could be attributed to the ineffectiveness of requiring 
freshman composition. Perhaps by the college freshman 
year, writing habits are so established that instruction 
can affect them but little. Or perhaps instruction did 
have a significant effect which was lost a year or more 
later when the posttest was administered, because the 
students had had inadequate challenge or opportunity to 
maintain the proficiency they had developed. In this 
experiment, however, two other conditions may have 
helped to produce small differences among papers: 



a. A mistake was made in removing all 
practical motivation for the students 
to write well. Some means should have 
been developed for motivating all of 
the students and motivating the 
Experimental and Control students 
equally. Suggestions for accomplishing 
this would be gratefully received by 
the authors of this report. 



b. Evidently in contrast to some of the 
other universities cooperating in the 
UNI experiment, the U of I emphasizes 
argumentative writing to a considerable 
extent. This is probably the most 
distinctive emphasis of the U of I 
program in contrast to the high school 
composition programs from which its 
freshmen come. Consequently, the 
smaller U of I study erred in using 
expository assignments merc^ly because 
the larger UNI experiment cid. If the 
U of I freshmen had in fact written much 
less argument than exposition in high 
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school, then the U of I instruction 
in argument should make much more 
difference in their writing argument 
than it would in writing exnosit ion. 
Probably the smaller U of I experiment 
(aniperhaps the larger UNI study also) 
was testing the wrong thing. 



Although the Cooperative Research Program grant did not 
obtain the answers which the study sought, it may well have taught 
the investigators (and readers of this report) how to conduct a 
similar study in a much more valuable way. Such a modified 
replication is recommended with the changes above. Two other 
suggestions are made as a part of this recommendation: 

1. Certain modifications are currently being 
made in the course of study of the Rhetoric Program and 
in the advisory system it maintains for its graduate 
assistants. These changes should be completed and 
working satisfactorily before another experiment of 
this type is undertaken. One should test the ef<^ec- 
tiveness of the best he can do. 

2. Probably the real alternative to the freshman 
Rhetoric requirement is not the complete absence of any 
instruction in writing, public speaking, or reading. 

Although a new experiment should still have a group of 
students excused from everything, two other experimental 
groups should be used to represent practical alternatives 
to the existing requirement: 

a. A group from the total experimental 
group which elects to take writing or 
Rhetoric at the junior level instead 
of the freshman level. 

b. A group from the experimental group which 
takes public speaking but no instruction 
in writing except as individuals who 
periodically report to the Writing 
Laboratory to review on an individual 
basis papers they have written for other 
courses or at their own individual 
initiative. 
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APPENDIX A 



LETTERS TO STUDENTS 
—AND ONE RESPONSE 
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STATi: 

UNIVERSITY OF IOWA 

IOWA CITY, IOWA 




<e of Liberal Arts 
of the Dean 



July 20, 196A 



Dear Student: 

In cooperation with four other colleges and universities and with the 
strong support of the U.S. Office of Education, the University of Iowa is 
conducting an experiment to test the effectiveness of college instructlo 
in freshman composition. To conduct this experi^nt, we must ^ 

performance of students who take Rhetoric with those who <lo ^ou have 

Len selected as a member of the group of 331 students at SUI who 
be required to take Rhetoric. I hope that you wl.-.l cooperate to make th 

Important research meaningful. 

Your cooperation in the following matters is essential to the success 
of the experiment: 

1. When you register at the Field House this September, register for 

the Rhetoric course numbered 10:6, a course existing on paper 
only and bearing no credit hours. By registering for 10:6 you 
inform us that you are cooperating with the experiment and are 
not taking a Rhetoric course. If you register ^ , 

semester during the academic years 1964-65 and x965-66 (a total 
of four semesters), if you take no other composition courses 
during this period, and if you take the examinations described 
below, you will be excused from the Rhetoric requirement. 

2. Report at the following times to take the examinations, 
bringing a bail-point or fountain pen with which to write. 

These examinations take precedence oyer aJA othe r events . 



Tue., Sept. 15, 8:00-10:00 a.m. 
Wed., Sept. 16, 3:30-5:30 p.m. 
Thu., Sept. 17, 7:00-9:00 p.m. 



For each of these tests 
you are to report to: 



Theme: 

Objective Tests: 

Theme: 

You will be asked to report again in January, 1965, and in June, 1965, 
to take alternate forms of the objective tests and of °ne of the theme 
examinations. Then you will be asked to report in June, 1966, to take all 
three examinations again as you will have done 

enroll for 10:6 and take these examinations, you will be permitted to 
graduate without taking Rhetoric. Although your graduation from college 
will be in no way dependent on how well you do on them, we expect you to 
do your best work on the examinations. If you do not take the examinations, 
howLer, you will be required to enroll in the appropriate Rhetoric course 

at your first opportunity. 



er|c 
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Most students being excused from Rhetoric probably will wish to register 
for one of the Literature Core courses instead, though you may, if you wish, 
confer with your adviser about taking another course. If you have any questions 
about the experiment now or after you enter the University, please write or 
see Professor Richard Braddock, Coordinator, Rhetoric Program, Old Armory 

Xt2lupOJ.ai.7 i^Uj. xUi.a5 • 

So that we will know you have received this letter and understand it. 



Special Note for ”10:3 or 10:6” Students 

If ”10:3 or 10:6” is Indicated on your Final Admission Statement, you 
would normally be eligible to attempt to test out of Rhetoric. Your initial 
participation in the experiment will not deny you this privilege. When you 
write the Tuesday theme for the experiment, tell your proctor you wish ^e 
theme to count for the 10:3 theme test as well as for the experiment. Then 
you need not take the regular 10:3 theme test at 10 a.m. that morning. You 
will, however, still need to take the other examinations for the experiment 
and the 10:3 speech test scheduled on page 39 of the Handbook for 
University Students . If you achieve high enough scores on both 10:3 
examinations, you will be exempt from the Rhetoric requirement and from 
further cooperation with the experiment. You may find out whether or not 
you achieved high enough scores by inquiring at the Rhetoric Office on 
Tuesday, September 22. 



please sign the enclosed post card and return it before July 31. 

Yours sincerely. 



Richard Braddock 
Coordinator, Rhetoric Program 







Dewey B. Stuit 
Dean 



/ 
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UNIVERSITY OF IOWA 



IOWA CITY, IOWA 



STATE 




College of Liberal Arts 
Office of the Dean 



July 31, 1964 




Dear Student: 

In cooperation with four other colleges and universities and with 
the strong support of the U.S. Office of Education, the University of 
Iowa is conducting an experiment to test the effectiveness of college 
instruction in freshman composition. To conduct this experiment, we 
must compare the performance of students whp take Rhetoric with those 
who do not. You have been selected as a member of the group of some 750 
students who will take Rhetoric as usual but who will also take a series 
of examinations which will permit comparison of their writing performance 
to the performance of students not taking Rhetoric. Your group is 
designated as "the control group." 

Your cooperation will involve taking some special examinations during 
the course. of the two-year experiment. Please report at the following 
times to take the initial examinations, bringing a bail-point or fountain 
pen with which to write. These examinations take precedence over a 11 
other events . 

Theme: Tue., Sept. 15, 8:00-10:00 a.m. For each of these tests 

Objective Tests: Wed., Sept. 16, 3:30-5:30 p.ra. you are to report to: 

Theme: Thu., Sept. 17, 7:00-9:00 p.ra. 

By Christmas, 1964, you will be notified whether or not you are to 
continue being a member of the control group. If you are to continue, you 
will be asked to report again in January, 1965, and in June, 1965, to take 
alternate forms of the objective tests and of one of the theme examinations. 
Then you will be asked to report in June, 1966, to take all three examina- 
tions again as you will have done this September. Although the results will 
not be used in determining your Rhetoric grade, we must insist that you 
take the examinations as part of your regular Rhetoric requirement and 
we expect you to do your best work on them. 



o 



We are confident that you will be pleased to be one of the thousands 
of students in the five universities cooperating to make this experiment 
yield significant information. If you have any question about your part, 
'Pool t-o fri PirnfoccoT" B^sddock* Cooirdlnfiitoir • 

Rhetoric Program^ Old Armory Temporary Building. 

So that we will know you have received this letter and understand it, 
please sign the enclosed post card and return it before August 15. 



Special Note for 10:3 Students 

If your Final Admission Statement indicates that you should take the 
Rhetoric course numbered 10:3, you are eligible to attempt to test out 
of Rhetoric. Your initial participation in the experiment will not deny 
you this privilege. When you write the Tuesday theme for the experiment, 
tell your proctor you wish the theme to count for the 10:3 theme test as 
well as for the experiment. Then you need not take the regular 10:3 theme 
test at 10 a.m. that morning. You will, however, still need to take the 
other examinations for the experiment and the 10:3 speech test scheduled 
on page 39 of the Handbook for New University S tuden ts . If you achieve 
high enough scores on both 10:3 examinations, you will be exempt from the 
Rhetoric requirement and from further cooperation with the experiment. 

You may find out from your 10:3 instructor by Tuesday, September 22, 
whether or not you achieved high enough scores. 



Yours sincerely 



Richard Braddock 
Coordinator, Rhetoric Program 




fi- 



Dewey B. Stuit 
Dean 



CoUege of Liberal Arts 
Office of the Dean 
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STATE 

UNIVERSITY OF IOWA 

I OWA CITY, IOWA 

January 6, 1965 




We wish to thank you for taking the examinations for the Rhetoric 
experiment last September, Your cooperation^ along with that of the 
other students here at the University of Iowa and the other cooperating 
universities, will help insure that we obtain results which will be 
meaningful to higher education throughout the United States. 

Although some students are being dismissed from the experiment at 
this time because they could not be matched v;ith others, we have been able 
to match you with another student and hence are relying on you to continue 
in the experiment. We ask that you give the few hours each semester that 
it takes to participate in this important project and hope that you will 
do your best work in the examinations. 

In order not to inconvenience you by holding the examinations at night 
or on a week end, we have scheduled them during examination week as shown 
below. As there is no preparation for the experiment exams, we hope they 
will not seriously interfere with your regular examinations, especially 
when you know about them this far ahead and can plan your studying 
accordingly. The examinations for the Phetoric experiment take precedence, 
over all other events . Please note now on your calendar the time and 
place of each exam so that you will be present on time. 

For each of these tests 

Theme:, Mon., Jan. 25, 1:00 - 2:50 p.m, you are to report to: 

Objective Tests: Thu., Jan, 28, 8:00 “ 9:50 a.m. 

Please also note that you must continue to register for 10:6 each 
semester (except the summer session) through spring, 1966 . That way we can 
be sure to see that you are excused from the Rhetoric requirement. If a 
student in the experimental group neglects to take any of the experiment 
exams, he will be dismissed from the experiment and be required to fulfill 
the Rhetoric requirement like any other freshman. 

If you have any question about your status in the experiment or in 
Rhetoric, please do not hesitate to telephone or see Professor Braddock or 
his secretary, 4A OAT, Ext. 2229 or Ext. 2069, Again, thank you for helping 
make the Rhetoric experiment meaningful. 

N.B. Please bring a pen or ball 

point to' the theme exam, and 
a #2 pencil to the objective 

exam. Coordinator, Rhetoric Program 

Dewey B. Stuit 
Dean 
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College of Liberal Arts 
Office of the Dean 



STATE 

UNIVERSITY OF IOWA 

IOWA CITY, IOWA 




January 6; 1965 



We wish to thank ydu £6r taking the examinations for the Rhetoric 
experiment last Septembfer, Your cooperation, along with that of the other 
students here at tha Uhivdraity of Iowa and the other cooperating universities, 
will help insure that we obtain results which will be meaningful to higher’ 
education throughout the United States. 

Although some students are being dismissed from the experiment at this 
time because they could not be matched with others, we have been able to 
match you with another student and hence are relying on you to continue in 
the experiment. We ask that you give the few hours each semester that it 
takes to participate in this important project and hope that you will do 
your best work in the examinations. 

In order not to inconvenience you by holding the examinations at night 
or on a week end, we have scheduled them during examination week as shown 
below. As there is no preparation for the experiment exams, we hope they 
will not seriously interfere with your regular examinations, especially 
when you know about them this far ahead and can plan your studying accordingly. 
The examinations for the Rhetoric experiment take precedence over all other 
events . Please note now on your calendar the time and place of each exam 
so that you will be present on time. 

For each of these tests 

Theme: Mon., Jan. 25, 1:00 - 2:50 p.m. you are to report to: 

Objective Tests: Thu., Jan. 28, 8:00 ~ 9:50 a.m. 

To compensate you for the added writing you are doing for the experiment, 

your Rhetoric instructor has been asked to excuse you, if you wish to be 
excused, from one of his regular theme assignments in January (but not the 
final theme). At the same time, if a student neglects to take either 
experiment exam, he will receive an "Incomplete" for his final Rhetoric grade 
this semester. 

1 

If you havfe ahy Iqii6btl6n aljdUt your status in the experiment or in 
Rhetoric, please db ribt hesitate to telephone or see Professor Braddock or 
his secretary, 4A OAT, Ext. ^229 or Ext. 2069. Again, thank you for helping 
make the Rhetoric experiment meaningful. 

t • I 

N.B. Please bfih^ a lifeb of ball 

point to the theme bkam, And 
a #2 pencil t® the objective 
exam . 



Yours sincerely. 




Richard Braddock 
Coordinator, Rhetoric Program 

Dewey B.^Stuit 
Dean 
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THE UNIVERSITY OF IOWA 

* 



IOWA CITY, IOWA 52240 




Ma\r 1 0 - 1 Q6 5 



College of Liberdi Arts 
Office of the Dean 



We wish to thank you again for cooperating with the Rhetoric experiment. 
As we conclude the first year of the two-year project and information about 
the experiment spreads, we find many colleges and universities in the United 
States awaiting the outcome. Consequently, we hope that you will continue to 
do your best when you take the examinations this month and again a year from 



Because more students than anticipated found it inconvenient to take the 

experiment exams during Examination Week this January, we have scheduled the 

coming exams for the weekday evenings shown below. Do remember that the 

examinations for Rhetoric experiment take precedence over aU other eve n t s . 

Please note now on your calendar the time and place of each exam so that you 

will be present on time. , 

For each of these tests 

Theme: Mon., May 17, 7:00-8:50 p.m. you are to report to; 

Objective Tests: Thu., May 20, 7:00-8:50 p.m. 

If a student in the experimental group neglects to take any of the 
experiment exams, he will be dismissed from the experiment and be required to 
fulfill the Rhetoric requirement like any other student. Please note that 
you must continue to register for 10:6 during the fall and spring semesters 
next year (but not this summer). 

If you have any question about your status in the experiment or in Rhetor! 
please do not hesitate to telephone or see Professor Braddock or his secretary, 
4A OAT, Ext. 2229 or Ext. 2069. Again, thank you for helping make the Rhetorxc 

experiment meaningful, 

N, B. Please bring a pen or ball Yours sincerely. 



now. 



poitit to the theme exam and 
a pencil to the objective 
tests. 




Richard Braddock 
Coordinator, Rhetoric Program 






Dewey B. Stuit 
Dean 




I 



College of Liherd Arts 
C^e of the Dean 



THE UNIVERSITY OF IOWA 

lOWA CITY, IOWA 52240 




May 10, 1965 



We wish to thank you again for cooperating with the Rhetoric experiment. 
As we conclude the first year of the two-year project and information about 
the experiment spreads, we find many colleges and universities in the United 
States awaiting the outcome. Consequently, we hope that you will continue to 
do your best when you take the examinations this month and again a year from 

now. 



Because more students than anticipated found it inconvenient to take the 

experiment exams during Examination Week this January, we have scheduled the 

coming exams for the weekday evenings shown below. Do remember that the 

examinations for the Rhetoric experiment take precedence over othe r events,. 

Please note now on your calendar the time and place of each exam so that you 

will be present on time. , 

For each of these tests 

Theme: Mon., May 17, 7:00-8:50 p.m. you are to report to: 

Objective Tests: Thu., May 20, 7:00-8:50 p.m. 



To compensate you for the added writing you are doing for the experiment, 
we have asked your Rhetoric instructor to excuse you, if you ask him to, from 
the final reading examination in Rhetoric, which all other 10:2 and 10:3 
students will be required to take. Please remember, though, that if a^^student 
neglects to take either experiment exam he will receive an "Incomplete for 
his final Rhetoric grade and be expected to take the final reading exam with 
the Rhetoric students not in the experiment. 



If you have any question about your status in the experiment or in Rhetoric, 
please do not hesitate to telephone or see Professor Braddock or his secretary, 
4A OAT, Ext. 2229 or Ext. 2069. Again, thank you for helping make the Rhetoric 
experiment meaningful . 



N. B. Please bring a pen or ball 
point to the theme exam and 
a #2 pencil to the objective 
tests. 



Yours sincerely, , 

Richard Braddock 
Coordinator, Rhetoric Program 



Dewey B. Stuit 
Dean 
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THE UNIVERSITY OF IOWA 

IOWA CITY, IOWA 52240 




College of Liberal Arts 
Office of the Dean 



r- *1 r\K n 

hay D; 



Dear Student: 

Thanks to your cooperation, the Rhetoric experiment is nearing completion. 

This summer the five cooperating colleges and universities will be analyzing 
the results for a report in the fall. We look forward to informing you of 
the general result^ and of your own personal scores. 

You may recall' that we began at the U of I with 331 matched pairs of 
students. Drop-outs and transfers from the University and dismissal from the 
experiment have reduced our number to fewer than 100 pairs. As a ^®su t, we 
must insist that you appear for this last series of examinations, if you are 
in the experimental group (which was excused from Rhetoric), please remember tha 
we must hold you for the Rhetoric requirement if you do not complete these 
examinations. If you are in the control group (which took 

remember that you will be considered not to have completed ^ ’ 

ment if you do not complete these examinations. We hope you find that ® 
have been scheduled at convenient times for you; if you see some difficulty in 
this respect now, see Professor Braddock or his secretary in 4A OAT. 

Please report at the following times to take the examinations, bringing a 
pen or ball point to the theme exams, a #2 pencil to the objective b«sts. 

These examin a tions take precedence PVfr ajl pthgp pyeiyt a. ® 3 

Selective Service classification exam does not begin until 1 p.ra. on June J. 

For each of these tests 
you are to report according 
to the following schedule; 
Last 2 digits Room No. 
of Univ. No, 

00 - 33 121 -A SH 



Theme; 


Wed., May 25, 


7-9 p.m. 


Theme; 


Fri., June 3, 


7 ; 30”9 ; 30 a ,m. 


Objective 

Tests; 


Fri,, June 3, 


< 

9; 30- 11; 30 a.m. 



"\L - 66 



O01 



Yours sinc^el^ / 

Dewey Bq Stuit 
Dean 






Richard Braddock 
Coordinator, Rhetoric Program 




I 
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4A O.A.T. 

May 20, 1966 



REMINDER 



Don't forget the last battery of the Rhetoric experiment . 
examinations! 



Wed., May 25, 7-9 p.m. 

Theme. Bring pen or ballpoint. 

Fri. , June 3, 7:30-11:30 a.m. 

Theme and objective tests. 

Bring #2 pencil and pen or ballpoint. 

Each time, please report punctually to the room following the 
last two digits of your student number: 



00-33 


121A 


SH 


34-66 


221A 


SH 


67-99 


321A 


SH 



You deserve much credit for cooperating with the Rhetoric 
experiment, even when — to make it meaningful --we had to require 
you and the others to participate. Dean Stuit and I do hope that 
you will carry your cooperation through to the end by doing your 
very best on these last examinations. Then we will be able to 
give you an accurate report of your own writing proficiency and 
have significant results to justify the effort you have invested 
in the experiment. 

As a modest token of our appreciation of your efforts, we 
will serve coffee and doughnuts to you and the other particpating 
students outside 121A SH from 7:00 to 7:20 a.m. on Friday, June 3. 
We wish that it were possible to show our appreciation in a more 
substantial manner, but the spirit is there. 

Richard Braddock 

' Coordinator 

Rhetoric Program 




University of Iowa 
Iowa City, Iowa 52240 
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I4ay 18, 1966 



i'rofessor Braddock 
4A OAT 

Bear Sir: 

I am a sophomore who was selected for the 
Rhetoric experimental control group* I have here 
a letter informing me that I am to report for 
four hours of testing during final week and two 
hours of testing durin*-^ the week; imm ^distely pre- 
ceeding final week, during which I also have final 
exams. I have never understood why I was chosen 
to be in the control group or why I was nev ;r 
given any choice in the matter. I have completed 
the 10:3 program, completed all my other core require 
ments, aau. am well into my ma^or. Now I am informed 
that I must take Rhetoric exams during finals. 

With these exams hanging over my head, I rather 
doubt that my final exams will be up to par. 

Thus far I have taken these additional exams, 
done my best, and kept quiet; although growing 
more hostile towards the program with^^each set. 

In every letter received I have read, If you are 
in the control group ( which took Rhetoric ) please 
remember that you will be considered not to have 
completed the Rhetoric program requirement if you 
do not complete these examanitions. ” This serves 
as a constant reminder that I was forced to take 
these tests, I didn't volunteer. This has made 
me very angry. I also read in this particular 
letter that, "We hope you find that these exams 
have been scheduled at a convenient time lor you; 

I cannot believe that anyone who is realy sincere 



could make this statement v;hen he is I’uliy aware 
that he has oust assigned a set of exams which 
run from 7:50-11:30 a.m, during finals to college 






suuaemcs who WJLXX xcucavc a v/ow^U j 



from the exams. 



By this time you have no doubt received sev- 
eral letters such as this one. Nevertheless I 
do feel that I have a legitimate complaint and 
1 should lodge that complaint wiht the proper 
persons. If you feel that you have a legitimate 
explanation or point I am usually available at 

353-IOA.5. 

S^Lncereiy, 

Dennis Laitg 



cc : Dean Stuit 










ippil ipiUP^UI 1 IJJliJi 
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APPENDIX B 



RHETORIC EXPERIMENT QUESTIONNAIRE 



M 




ti- 1 f-ri[ irri'^iif lii 
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RHETORIC EXPERIMENT QUESTIONNAIRE 
May 25» 1966 



My Student Number: (Please make your numbers legiblco) 

Check One: 

( ) I was excused from the Rhetoric requirements (I was in the 
experimental groups) 

( ) I was not excused from the Rhetoric requirement <> (I was in the 
control groups) 

Be sure to reed ell the responses for each question before answering » 

Ir When you cao^to the U of I your first September « how well prepared were you to 
do the kinds of writing you have since done ^ courses other then Rhetoric? 
(Check only one responseo) 

( ) Ir. Fully preparedo The composition instruction in Rhetoric was not 

necessary for me<. 

( ) 2o Rather well prepared ^ Al though Rhetoric night have helped ae 

improve my writing someo I did not need to take Rhetoric to do 
adequate writing for my other freshman and sophomore courses.. 

( ) 3n Rather weakly preparedo Although my writing was not bad enough 

to have lowered my grades in other freshman :nd sophomore courses 
without having taken Rhetoric^ I would not want to leave my writing 
at tnat level. ^ 

( ) 4c Quite weakly preparedo My writing in other freshman and sophomore 

courses (a) would have been weak enough to lower seme of my grades 
if I had not had Rhetoric OR (b) was weak enough that it did lower 
some of my grades because T^did not have Rhetoric » 

( ) 5c The question does not pertain to meo (If you check 5, also check 

either a or b or both belowo) 

( ) fir I have done no or almost no writing (term papers, ossay tests, 
etCf) for other freshman and sophomore courses.- 

( ) bo Although my writing needed improvement, (a) Rhetoric did 
not help it improve OR (b) I did not take Rhetoric, but 
it would not have helped my writing improve o 



1 

4 



more on next 









. , UIU. n lliuiPP 
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2 « 



2 < Knowing what yoi: know now^ if you were an entering freshman at U of I and 
Rhetoric t»*as an elective, not a required subject, which of the following would 
you do? (Ciieck only one response r.) 



( ) 1. Not take Riietoric or any other freshman or upporclass college course 

offering instruction in expository and argumentative writingo 

i ) 2r Not .take Rhetoric as a freshman, but plan to elect, as an 

upperclassman, a two-hour course in expository and argumentative 
writings 

( ) 3c Hot take Rhetoric as a freshman, but plan to elect, as an 

upperclassman, a four»hour Rhetoric course (expository and 
argumentative speaking and writingo 

( ) 4o Take Rhetoric as a freshman unless, as under the system now 

extended to all freshmen who do well on the ACT tests, I was able 
during Registration Week to write a theme and deliver a speech 
which met the Rhetoric Program standards for exemption o 

( ) So Take Rhetoric as a freslinan even though I could exe^[}t aorself at 
the outset by exaninationo 

( ) 6a Take Rhetoric as a freshman in any event ’ plan to take, as 

an upperclassman, an additional two-hour c ? in expository 
and argumentative writingo 

So Which of the following writing work have you t<»Ken while at U of I other 

than the regular Rhetoric work? (Check either answer 7 or one or more of 

answers 1-6 o) 



( ) Ir At least four individual lessons in a correspondence course 

in composition r 

( ) 2o At least four individual sessions of voluntary, non-credit work 

in the Rhetoric Writing S^aboratoryc 

C } Sc Rhetoric 10; 9 „ 

( ) 4c One of the following courses: P,q}ository Writing, 

Beginning Reporting, Magazine Article Writing, 

* Bi;siness Writing, or Technical Writings 

C ) So Undergraduate Writer’s Workshop; Poetry^ 



( } 6^ Undergraduate Writ’^r’s Workshop: Fiction^ 



( ) 7a I have taken none of the work listed in answers l-6o 
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APPENDIX C 



INSTRUCTIONS TO STUDENTS 



WRITING TOEMES 



Theme #1 (U of I) 
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Print your name legibly here 



(Last name) 



(First) 



(Initial) 



Your student number 



Experiment in the Effectiveness of College Composition 

« . .1 ... 1 r \£. A 

aepcemoer, 



200 



Follow these directions carefully: 



3. 



4. 



5. 



O 

RIC 



Do NOT write your name, date or instructor’s name on the theme booklet cover or 
anywhere in the theme booklet, DO write ycur student number at the upper right* 
hand corner of the theme booklet cover and your name and number at the top of inis 
sheet. The ID number you use for this test is your 6-digit University ID number. 



2. Select one of these general subjects and write it on the cover of the theme booklet 



A Lesson Learned in Childhood 
A Historical Event Prior to 1960 



A Natural (Not Man-Made) Event or Object 
What Makes Beautiful (or Ugly) 



NOTE: As you plan your paper and write it, please avoid including anything in 

your theme which will date it. That is, the theme graders should not find any 
references to very recent events, your age, your educational status, or the like. 
Two years from now, when this theme and a theme you will write then are rated, 
the graders should not be able to determine which theme is which except by noting 
improvement of your thinking and writing. 



Narrow this general subject to a specific topic that you can explain adequately 
in two hours. Note that the writing is to be an expla nation, not an argument npx 

a narrative. 



Example: Suppose the general topic is "The Theater.” This can be 
limited until the topic actually used is something like one of the 
following: 



Why I Prefer Character Parts 
What Makes a Good Prompter 



Hamlet's Insanity 

The Form of the TV Western 



Outline the theme, using the back of the front cover. (The outline will not be 
graded.) 



Write the theme, beginning on the next page. If you think you have time, you 
may write a rough draft on the scratch paper provided and copy it into the 

theme booklet. 



a, 

b 



Be sure your theme is an explanation, not an argument nor a “larrative. 
Avoid merely "discussing" a topic; develop a clear, central idea. 



Develop your points with specific material: facts, examples, illustrations, 
concrete details, >even charts and graphs if relevant. 



Follow the organization of your outline. 

Write a substantial paper. In length it should be at least 450 words. 
Use standard English. 

Take two full hours. Do not try to cut the time short. 

Check your writing carefully for grammar, spelling, and punctuation. 

Turn in all material, including scratch paper. 



d. 

e. 

f. 

g. 



NOTE: Any student detected consulting another student or referring to material 

the e x a min at i o n r oo m w i l l — be repQX .. t . e . d— t . Q -- the _ Co ., l _ le g e 



I 



■1 






I 



I 
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Theme #2 (ETS) 


I 

1 
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1st grader ( ) 


Second grader ( ) 


1 First grader ( ) 




rl A 


! Grade — - 


iJnci grader \ } 


VjrxcUXt? ‘ — 


1 

Student number 


X 0 xai — - 


1 _ _ 




THEME INSTRUCTIONS 


1085 



1. The paper which you are about to write will be judged on your succe^ in presenting your j 

thoughts in a clear, unified, well-organized manner, observing the conventions of standard writ- \ 

ten English. You should think about the topic until you have determined what idea you want to 
convey to the reader and the general procedure you will follow in doing so. Then you may write 

your paper. Do not hesitate to make a brief outline if you desire to do so (use the back of this . 

sheet) . An outline is not required. 

2. You should write as neatly and legibly as you can, but you should not hesitate to make changes 
between the lines if you believe them to be necessary. You do not have to copy the paper over. 

3. WRITE ON ONE SIDE OF THE PAPER ONLY. If you need more paper, ask for it. 

4. Begin on the third line of the first sheet, and WRITE ON EVERY LINE THEREAFTER. 

5. You must write with INK or BALL-POINT PEN. 

6. Be certain to write your STUDENT NUMBER in each of the blanks (two at the top, one at the 
bottom) provided for it on this sheet, and in the upper right-hand corner of each page of your 
theme. 

1 

7. Turn in all of the paper given to you. 



8. You must stay at least one hour and fifteen minutes. 

9. LENGTH ; 300 - 500 words. 



TOPIC 

Today a young man who' wears a beard or a girl who prefers slacks to skirts has difficulty in find- 
ing employment in most work which serves the public. Changes in fashion are announced one day 
and adopted the next. In business, promotions are made with great emphasis upon how well an in- 
dividual meets the “image” the employer wishes to create. In school, those who do as they are 
told and give the answers expected of them are rated high by many of the faculty ; those who do 
what “everyone else” does are popular with the students. 

Now consider a famous quotation i “Whoso would be a mar must be a non-conformist. 

Relate the material in the opening paragraph to the quotation, indicating whether, on the basis of 
your observation and experience, you feel the idea expressed in the quotation is true. 



Student Number 

College or University 
Your Name 



1 







last 



first 



middle initial 






Print your name legibly here 

(Last name) 

Your student number 
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f 506 ’ 

(First) (Initial) I 




Follow these directions carefully: 1 

?. 

1. Do NOT write your name, date or instructor's name on the theme booklet cover or « 

anywhere in the theme booklet, DO write your student number at the upper right- 
.»and corner of the theme booklet cover and your name and number at the top of this I 

sheet. The ID number you use for this test is your 6-digit University ID number. 



2. Select one of these g eneral subjects and write it on the cover of the theme booklet. 

A Lesson Learned in Childhood A Natural (Not Man-Made) Event or Object 

A Historical Event Prior to 1960 What Makes Beautiful (or Ugly) 

NOTE: As you plan your paper and write it, please avoid including anything in 

your theme which will date it. That is, the theme graders should not find any 
references to events of the past three years, your age, your educational status, 
or the like. Next month, when this theme and the themes you v/rote two years 
ago are rated, the graders should not be able to determine which theme is which 
except by noting improvement of your thinking and writing. 

3. Narrow this general subject to a specific topic that you can explain adequately 

bwo hour s . Note that the writing is to be an explanation, not an argument 
nor a narrative . 

Example: Suppose the general topic is "The Theater." This can be 

limited until the topic actually used is something like onr of the 
following: 

Why I Prefer Character Parts Hamlet's Insanity 

What Makes a Good Prompter The Form of the TV Western 

4. Outline the theme, using the back of the front cover. (The outline will not be 
graded.) 

5. Write the theme, beginning on the next page. If you think you have time, you 
may write a rough draft on the scratch paper provided and copy it into the 
theme booklet. 

a. Be sure your theme is an explanation, not an argument nor a narrative. 

b. Avoid merely "discussing" a topic; develop a clear, central idea. 

c. Develop your points with specific material; facts, examples, illustrations, 
concrete details, even charts and graphs if relevant. 

d. Follow the organization of your outline, 

e. Write a substantial paper. In length it should be at least 450 words. 

f. Use standard English. 

g. Take two full hours. Do not try to cut the time short. 

6. Check your writing carefully for grammar, spelling, and punctuation. 

7. Turn in all material, including scratch paper. 

NOTE: Any student detected consulting another student or referring to material brought 

l n . t o the examina t i o n c o am w i l l ■ ba- r a o o r t ad t o t h o C oIl e a e- Di s Gipl - in a rv Bo a rd. 
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TOTAL SCORE 
Student Number: 





Second Reader 


Score 


First Reader 


score 




(# ) 




(# ) 





i 

t 

I 



f 



THEME INSTRUCTIONS 



1 . 



The paper which you are about to write will be judged on your success in presenting youf 
thoughts in a clear, unified, well-organized manner, observing the conventions of standard writ- 
ten English. You should think about the topic until you have determined what idea you want to 
convey to the reader and the general procedure you will follow in doing so. Then you may write 
ydur paper. Do not hesitate to make a brief outline if you detire to dp so (use the back of this 
sheet) . An outline is required. 



2. You should write as neatly and legibly as you can, but you should not hesitate to make changes 
between the lines if you believe them to be necessary. You do not have to copy the paper over. 



3. -WRITE ON ONE SIDE OF THE PAPER ONLY. If you need more paper, ask for it. 

4. Begin on the third line of the first sheet, and WRITE OK EVERY LINE THEREAFTER. 

5. You must write with INK or BALLrPOINT FEN. 



6. Be certain to write your STUDENT NUMBER in the blank provided at the top of this instruc- 
tion sheet in the upper left-hand comer under the Total Score box. It should also be written on 
each page of your theme. Do NOT write your name, or the name of your scho ol, in any pl*<=g 
other thw" the blank provided at the bottom of thb aheet. 



7. Turn in ail of the paper given to you. 

8. You must stay at least one hour and fifteen minutes. 
9i LENGTH : 300 - 600 words. 



TOPIC 



Conventional is a word frequently used to refer to customary attitudes, beliefs or actions. In the 
Un-ted States it is a convention for men to be clean-shaven, women to wear a certain amount of make- 
up, boys to be interested in sports, and girls to be interested in becoming wives and mothers. A 
person who is unconventional in some way departs from the conventions of action or belief of the 

society of which he is a part. 



With this explanation in mind, discuss the following statement: 



“Convention is society’s safeguard, but also its potential executioner.” To what extent and 
in what ways do you agree with this statement? Use examples and details from your knowledge and 
experience to support your conclusion. 




(Last Name) 



(First Name) 



(Middle Initial) 
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INSTRUCTIONS TO RATERS 
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lo Your purpose should be to rate each paper as accurately as you can^ 
without attenpting in any way to detenoine who wrote the paper or 
when it was written o Rate the experiment themes just as you would 
final theme examinations in the Rhetoric Program except for the 
following deviations 0 

2» MAKE NO MARKS ON ANY OF THE THENESo 

Confine all your notes and marks to scratch paper and theme score 
sheets until you are ready to place your ratings on the cover sheet 
of the theme bookleto Make no narks whatsoever inside the thone 
booklet except for a faint star at the end of the line in which 
you estimate the 400th word to fallc 

3o Disregard the Rhetoric Program letter grade systeao That is, rate 
merely to assign numbers; do not give a **14F,” for instance, if a 
theme merits a ”1” in some category^ 

4c Reconcile each of the five category ratings so that you and your 
rating partner are never more than one point apart <. Reconcile the 
total score so that you are never more than two points apart » 

THEN ADD YOUR TWO TOTAL SCORES TO YIELD A SUM, not an average total* 
This sum must be a whole number, with no fractions or decimals « 

(Thus if your total was 15 and your partner's was 17, your total 
rating would be 32») Be sure to check your addition after you enter 
your ratings on the cover sheet » 

So Inasmuch as the students had no access to dictionaries when they wrote 
these themes, deduct only two points under "Literacy” for each 
misspelled word the first time it is misspelledo (As usual, deduct 
only one point for a dear error in hyphenating or compounding words <> 

As usual, do not count a variant spelling as an error; if in doubt, 
consult a list of variant spellings which Mr« Braddock haso) 

6o Although you may consult Mr<> Braddock about hypothetical, graoDiaticaX 
and mechanical problems if you feel the need, do not ask him to resolve 
differences of opinion about the rating of a category on any specific 
theme o He should not be permitted to affect your ratings and hence, 
though unintentionally, to bias the experiment « 



7« If one of the theses written in an Examination Service booklet reveals 
clearly whether it was written this spring or two years ago, rate the 
theas» and reconcile your scores as usual, without mentioning the date 
to your rating partner. After you have completed your ryonciling, 
ask your partner whether or not ne recognized the date of wri^ng. 

Then bring the theme to Mr. Braddock. 

8e Instruction sheets for the themes are attached. Note that two of the 
instruction sheets differ from the kind usually employed in the 
Rhetoric Programo Use the same procedures in rating each type of 
theme, however. 



Rhetoric Program 
Ihiiversity of Iowa 
June 8, 1966 
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INSTRUCTIONS FOR STANDARD RATING OF THEMES 

To achieve as Ooiiaisteait theme stating as possible, the Rhetoric 
Program has developed a standard method of grading which is used for all 
placement and final themes and which instructors use frequently during the 
semester. The method consists of analyzing a theme into five major aspects 
—central idea and analysis, supporting material, organization, expression, 
and literacy. By rating each of these categories from 1 to 5 (F to A) and 
totalling the five ratings, the instructor sees ’’the whole theme" as well 
as its various aspects. Periodic practice rating sessions of the entire 
Rhetoriv faculty further develop consistency from instructor to instructor. 

A discussion of each of the five major aspects follows. 

Central Idea and Analysis 

The "C" paper clearly states or implies a reasonably restricted thesis 
or proposition on a significant subject which is appropriate to the assign- 
ment. The purpose of the theme is expository, argumentative, or critical, 
as the assignment requires, and is not principally narrative or descriptive. 
If otherwise the pertinence would not be clear, the introduction shows 
the relation of the central idea to the assigned topic. Once the narrowed 
central idea is made clear, usually by the end of the introduction, the 
theme continues to focus on that idea throughout the paper. Key terms 
are defined when necessary. The body of the paper is clearly divided into 
an appropriate number of significant steps, reasons, issues, or other 
considerations of approximately equal or ascending importance which lead 
the reader to understand or even to accept the central idea. 

The "B" or "A" paper presents a central idea which is especially 
challenging or significant for freshmen. The analysis reflects superior 
understanding of the issues or other considerations involved and, in the 
best papers, relates these meaningfully and interestingly to the concerns 
of the reader. 

The "D" or "F" paper may deal with a subject which is trite or 
inconsequential even for freshmen. It presents a main idea which is vague 
or too broad, or it wanders from one thread of a topic to another without 
weaving them into a unified pattern of development. Frequently the paper 
contains a number of paragraphs only two or three sentences in length. 

The theme may seem to have been written merely to fulfill the assignment, 
not to communicate. If the theme clearly does not fulfill a major 
consideration of the assignment, the paper does n^t deserve to pass. 

Supporting Material 

The "C" paper explains or supports its general statements with enough 
relevant facts, figures, specific instances, quotations, or other details 
to make the paper clear. The supporting material may be drawn largely from 
the student's experience, or it may be obtained by reading or other means. 
After it has been made clear that a student is expected to acknowledge the 
sources of his information and quotations, he does this, either by footnotes 
or by informal mention in the text of the paper. 
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The "B" or "A" paper qualified its statements and supports them more 
concretely, perhaps even with fresh and interesting details. The best 
papers indica’-e the conclusiveness of supporting evidence, showing that 
an instance or illustration is typical, or that a sample is representative. 
If they present material which is surprising to the reader, they do so 
with an awareness that special explanation or reassurance is necessary. 

'ihe ‘'p'* or ”F” paper may contain a considerable amount of material 
which is irrelevant, overobvious- contradictory, or biased. Rathor than 
offer evidence, it may depend upon hypothetical example, forced analogy, 
or mere rhetoric. Note: Instructors are required to report cases of 

suspected or outright plagiarism to the Chairman of Student Relations. 

(See page 2.5.) 



Organization 

The ”C” paper employs an introduction, body, and conclusion, although 
in short papers the introduction and conclusion may not always be long 
enough to occupy separate paragraphs. The introduction makes clear the 
purpose or central idea of the theme. Each paragraph of the body presents 
or inqplies a clear topic sentence, and explains or supports the topic in 
a unified manner. Transitions from paragraph to paragraph are clear, though 
not always smooth. The conclusion restates and emphasizes the central idea 
without introducing new material or irrelevancies • 

The "B” or "A” paper introduces the central idea so that it engages 
the concern or interest of the reader. The paragraphs often have internal 
divisions and transitions, perhaps using such techniques as comparison and 
contrast, cause and effect, or pioblem and solution. The conclusion is 

emphatic, often suggesting the larger significance of the restricted 
central idea. 

The ”D" or "F" paper has little apparent pattern of development or 
one which depends upon such rudimentary transitions as "firstly," "secondly," 
and "thirdly" or an excessive use of introductory rhetorical questions. 

The introduction and conclusion often are terse to the point of abruptness . 



Expression 

In "Expression," the "C" paper shows evidence of being moderately 
concise varied in diction and sentence style without too many errors 
or infelicities. The indicated number of points for the following types of 
errors is totalled, and the sum is converted into the rating indicated 
in the scale below. 

Tvio points are added for each example of extensive wordiness, loosely 
strung out sentence, wrong word meaning, excessive cooi'dination, unnecessary 
shift in subject or verb, poor parallelism, dangling or squinting modifier, 
and unclear or illogical word order or subordination. 

One point is added for each example of trite diction, ambig Ity, 
unintended connotation, deadwood, ineffective repetition, and unidiomatic 
use of a preposition or other word. 
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Conversion Scald for *’ Expression'* 



Total Points 

per 400 words: 21 and over 



16-20 11-15 6-10 0-: 



t\ it • 



m 

V -y 



n f?'! 



C f31 



B f41 



A (5) 



The rating for "Expression" may be raised or lowered one step if the 
total points dc not seem adequately to reflect the rater's general impression 
of the expressiveness of the paper. 



Literacy 

"Literacy" is rated in much the same way as "Expression" except that 
special provision is made to prevent a paper .Trom being drawn into the 
"B" or "A" bracket merely because of an absence of gross errors. 

Add five points for each run-on sentence and each bad sentence fragment 
and comma splice. 

Add three points for each misspelled word the first time it is misspelled. 
Variant spellings ai5 not misspellings. (If in doubt, consult Donald 
Emory's Variant Spellings in Modern American Dictionaries , NOTE, 1958.) 

The three point penalty is”Fased on the assumption that the student has 
access to a desk dictionary. 

Add two points for each example of subject-verb disagreement, pronoun- 
antecedent disagreement, unclear pronoun reference, incorrect verb form, 
non-stylistic omission, and other strikingly ungrammatical construction. 

Also add two points for each clearly erroneous use of period, colon, 
semicolon, question mark, and quotation marks. 

Add one point for each error in hyphenation or compounding of words, 
capitalization, and punctuation not covered above, and for each example of 
carelessness. 



Conversion Scale for "Literacy" 

Total Poin ; 

per 400 words: 21 and over 11-20 0-10 

Rating: F (1) D (2) C (3) 

Certainly the adequate paper should be written or typed clearly, with 
reasonably neat margins, in a theme booklet or on appropriate paper. 
Appropriate paper is white bond (not onion skin) measuring 8 1/2" by 11" 
and not having the frayed edge of a page torn from a spiral binding. If 
out-of-class themes are not typed, they should be written in ink, never in 
pencil in the final draft. The final draft should be proofread carefully 
before it is submitted , last-minute corrections being neatly written in 
ink and excisions made with a single line. If footnotes or a bibliography 
is included , some acceptable and consistent style should be followed. 

If a manuscript deviates markedly from this form, its rating for "Literacy" 
is lowered one or two steps. 




Points are counted on placement and final themes (which must be rated 
by a deadline) only through the first 400 words. As one step in achieving 
consistent ratings, the following procedure is used in estimating this lengL*'. 

1, Count the total number of words (including short prepositions, etc.) 
in six lines of representative length distributed throughout the theme. 
Using the table here, find the approximate number of lines which 400 
words take in this theme. 



Total No. Wds. 
in Six Lines 


No. of Lines 
for 400 IVds. 


Words 


Lines 


Words 


Lines 


29 


83 


40 


60 


52 


46 


30 


80 


42 


57 


54 


44 


32 


75 


44 


54 


56 


43 


34 




46 


52 


5C 


41 


36 


67 


48 


50 


60 


40 


38 


63 


50 


48 


70 


34 



2. Beginning with the first line of the theme, count lines until you 
reach the number obtained from the table above. (Make adjustments if 
several partial lines are included at the ends of paragraphs.) Place a 
star (*) at the end of that line so you will know where to stop counting 
and so your rating partner will not have to duplicate the procedure. If 
the theme has fewer than 400 words, adjust the rating of each of the 
categories downward proportionally. 
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EXPERIENCE OF RATERS 



Sargent Bush. Princeton, AB in English; Iowa, ^tA in English. 
Two years of teaching in Rhetoric Program. Article 
published in Nineteenth-Centurv Fiction. 1965. 

Ellen Fox. Iowa, BA and MA in English. Two years of 
teaching in Rhetoric Program; no previous teaching 
experience . 

Benjamin Hodge. Princeton, AB in English; Iowa, MA in Speech 
and Dramatic Art. Two years of teaching in Rhetoric 
Program; two years teaching English in the Loomis 
School, Connecticut; two years teaching English as a 
second language in the Niamey Schools, Niger Republic. 

Nancy Holmes. Kansas, BA in English; Iowa, MFA in English. 
Two years of teaching in Rhetoric Program. Editor, 
Hallmark Cards, four years; free lance book reviewer, 
Kansas City Star , two years; six poems and a short 
story published in literary quarterlies; award in 
national poetry competition. 

Deborah Hawkins Horne. Iowa, BA in English. One year of 
teaching in Rhetoric Program; one year teaching 
American literature in LaFollette High School: two 
years teaching English and French, Southeast Junior 
High School, Iowa City. 

Paul Jensen. Columbia, BA in English; Union Theological, BD; 
Iowa, MA in English. Two years of teaching in Rhetoric 
Program; one semester of student teaching in English, 
Pacific Palisades High School. A sketch and a story 
published in literary quarterlies. 

Belden Johnson. Harvard, BA in American History and 

Literature. One year of teaching in Rlietoric Program; 
two years of private tutoring in English, American 
history, and mathematics at Harvard. Two stories and 
a poem published in undergraduate literary magazine. 

Michael Kelly. University of South Florida, BA in Speech and 
English. One year of teaching in Rhetoric Program. 
Editorial work on undergraduate literary monthly. 

Larry Kramer. Ohio State, BA and MA in English. One year 
of teaching in Rhetoric Program; two years of teaching 
freshman English at Ohio State; two years of teaching 
freshman English at Missouri. Several poems published 
by literary quarterlies; poetry prize at Ohio State. 
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Robert Miller. Tufts, BA in Drama. One year of teaching 
in Rhetoric Program. Wrote and edited for high 
school and undergraduate papers. 

Edward Pixley. Concordia College, BA in English and Speech; 
Wisconsin, MA in Speech and Drama. Two years of 
teaching in Rhetoric Program; one year of teaching 
public address at Wisconsin; four years of teaching 
speech and English at Waldorf College. 

Stanley Renner. Iowa, BA in English, all courses for Ph.D. 
in English. One year of teaching in Rhetoric Program. 
Hiree years of industrial journalism, two years as 
editor of house organ for 20,000 employees. Bell 
System in Chicago. 

Gary Stewart. Brigham Young, BS and MS in Drama; Iowa, 
all courses for Ph.D. in Speech and Dramatic Art. 

Three years of teaching in Rhetoric Program; one 
year teaching speech at Brigham Young; one year 
teaching humanities at Southern Illinois. 

Edwin Weihe. Brown, BA in English; Iowa, MA and MFA in 
English. One year of teaching in Rhetoric Program; 
one year teaching English, algebra and social studies 
at Maret School, Washington, D. C. Book reviews 
published in Studies in Short Fiction ; short stories 
published in undergraduate literary magazine. 
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ANALYSES OF THEME RATINGS 
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TABLE 8 
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MEANS OF BETTER SCORES ON THEME 3 OR 4* 



Ctotnl Idoa 
end Analysis 





E 


C 


row 


h 


6.50 


7.00 


■ 

6.75 


k! 


5.39 


5.74 


5.57 


4 


5.57 


5.07 


5.32 


col. 


5,49 


5.70 





Differences between levels 
significant at the .05 level 
with an observed F of 3.25. 



Supporting 

Ifoterial 

E C TOW 



h 


6.18 


1 1 

5.54 


5.86 


h 


6.05 


6.10 


6.08 


h 


6.40 


5.80 


6.10 


col. 


6.10 


5.98 





Nothing significant. 



Organisation 





E 






«-3 


6.20 


7.60 


6.90 


4 


5.90 


6.11 


6.00 


4 


6.07 


5.11 


5.59 


col* 


5.95 . 


6.02 





Interaction differences between 
E and C significant by levels at 
the .05 level with an observed 
F of 3.64. 





Literiicy 
E C 


row 


4 


5.80 


5.60 


5.70 


h 


5.34 


5.20 


5.27 


h 


' 5-00 


4.86 


4.93 


col. 


5.31 


5.17 





Nothing significant. 



Expression 



ts 


6.90 


7.80 


7.35 


h 


6.78 


6.56 


6.67 


h 


6.68 


6.46 


6.57 


col.. 


6.77 


6.62 




Nothing significant. 







Total Score 

E C 




h 


31.80 


33.80 


32.80 


h 


29.46 


29.71 


29.58 


h 


29.50 


27 . 04 


28,27 


col. 


29.61 


29.46 





Differences between levels 
significant at the .10 level 
with an observed F of 2.92. 



* An individual's score is the sum of the ratings of the two raters on 
the theme on which the individual made the better score. 



TABLE 9 
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MEANS OF SUMS OF SCORES ON TflEMES 3 AND 4* 



Gtntiral Idea 
and Analysis 





E 


C 


row 


Is 


11.10 


11.90 


11.50 




10.01 


10.09 


10.05 




9.68 


8.86 


9.27 


col. 


10.02 


9.99 





Nothing significant.' 



Simporting 

Material 





B 


C 


row 


^3 


11.00 


11.00 


11.00 




10.38 


10.56 


10.47 


w 


9.96 


9.32 


9.64 


col. 


10.34 


10.37 





Nothing significant. 



Qrganisatien 
H _ C ^ 





11.20 


12.40 


11.80 


h 


10.76 


10.93 


10.84 


h 


10.39 


9.64 


10.02 


col. 


10.7? 


10.80 





Differences between levels 
significant at the ,05 level 
with an observed F of 2.78. 



Expression 

E , C r-JSOL 



LS 


13.00 


13.40 


13.20 


LZ 


12.73 


12.36 


12.54 


h 


12.07 


12.29 


12.18 


coi. 


12.63 


12.41 





Nothing significant. 



Literacy 



Total Score 



h 

col 



E 


C 


TOW 






— C 




11.40 


11.10 


11.25 


ts 


57.70 


57.80 


58.75 


10.43 


10.02 


10.22 




54.29 


53.96 


54.13 


9.71 


9.79 


9.75 


h 


51.82 


49.89 


50.86 


10.36 


10.04 




col. 


54.07 


53.60 





Nothing significant. 



Differences between levels 
significant at the .10 level with 
an observed F of 3.07. 



♦ An individual’s score is the sum of the ratings of the two raters 
on each of the two themes. 
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MEAN GAINS BETWEEN BETTER THEMES* 



Cantfftl IdM 
and Analyjis 





E 


C 


row 


*•5 


«7A 

/U 


At OA 
4.x • UV 


4.x • 4.^ 


h 


20.33 


20.75 


20.54 


h 


20.71 


20.07 


20.39 


col. 


20.42 


20.70 





Nothing significant. 



Si^porting 

Matarial 





E 


C 


row 


1 

i*3 


- 

90 AO 


21 .v,0 


•yn Rft 


Lg 


20,70 


20.88 


20.79 


4 


21.11 

_ ^ 1 


20.64 

- -- 


20.80 


col. 


20.77 


20.85 





8 

Nothing significant. 



Organiaation 





E 


C 


row^ 


ts 


21.50 


20.14 


20.82 


h 


20.68 


20.69 


20.68 


h 


20.60 


21.60 


21.10 


col. 


20.82 


20.65 





Interaction differences between 
E and C significant by levels at 
the .10 level with an observed 
F of 2.95. 



Exprastion 



ts 


20.30 


21. 4C 


20. S5 


t2 


20.26 


20.12 


20.19 


h 


21.21 


20.32 


20.77 


col.. 1 


20.43 


20.23 





Nothing significant. 



Literacy 



Total Score 



ts 


21.00 


20.20 


20.60 




19,98 


20.13 


20.05 


h 


20.11 


20.29 


20.20 


col. 


20.06 


20,17 





Nothing significant. 



4 


23.20 


26. OC 


24.60 


Lg 


21.94 


22. 5f 


22.26 


h 


24.64 


21. 4e 


23.05 


col. 


22.50 


22. 6( 





Nothing significant. 



*An individual’s score is the difference between the better posttest theme 
and the better pretest theme. A constant of 20,00 has been added to each score. 
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TABLE 11 



MEAN GAINS BETWEEN THEMES 1 anJ 3* 



Central Idea 
and Analysis 





E 


V 


row 


! 


1 20.90 


21,30 


21.10 


h 


20.26 


20.73 


20.49 


k 


21.32 


20.93 


21.13 


col. 


20.49 


20.80 





Nothing significant. 



Organisation 
g . C rott 



h 


20.60 


20.60 


20.60 


h 


20.47 

1 


20.78 

- 


20.62 


h 


20.96 


20.64 


20.80 


col* 


20.56 


20.75 





Nothing significant. 



Literacy 





E 


C 


row 




21.00 


19.40 


20.20 


h 


20.13 


20.16 


20.14 


4 


20.21 


20.21 


20.21 


col. 


20.20 


20.12 





Nothing significant. 



Siqiporting 

Matarial 





B 


C 


row 


1 

*•5 




20.80 


20.70 


h 


20.38 


20.97 


20.68 


h 


20.68 


20.71 


20.70 


col. 


20.45 


20.91 





Nothing significant. 





Expression 

E C 


row 




[ 20.30 


20.00 


20.15 




20.40 


19.^8 


20.19 


4 


20.64 


20.61 


20.63 


col. 


20.44 


20.10 





Difference between E and C 
significant at the .05 level with 
an observed F of 4.27. 



Total Seoro 



h 


23.40 


22.10 


22.75 


h 


21.63 


22.62 


22.13 


4 


23.82 


23;il 


23.46 


col. 


22.13 


22.67 1 


- 



Nothing significant. 



* An individual's score is the difference between the two themes. A 
constant of 20.0 has been added to each score. 
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TABLE 12 

MEAN SUBTOTALS OF FIRST THREE CATEGORIES: 
Central Idea and Analysis, 

if 

Supporting Material, and Organization 



Means of Better Scores 
on Theme: 3 or 4 



«« 


Q 

19.10 


20.40 


19.75 




17.53 


18.00 


17.76 


h 


17.82 


15.71 


16.77 


col« 


17.68 


17.75 





Nothing significant. 



Mean Gains Between 
■Better Taemes 



e c _X9K 



**3 


21.90 


24.40 


23.15 




21.82 


22.33 


22.07 


h 


23.18 


20.86 


22.02 


col. j 22.06 22.20 
Nothing significant. 





Means of Sums of Scores 
on Themes 3 and 4 

B C nm 



Is 


33.30 


35.30 


34.30 


tj 


31.14 


31.58 


31.36 


h 


30.04 


27.82 


28.93 


col. 


31.08 


31.15 





Nothing significant. 



Mean Gains Between 
Tliemes 1 ana 3 



E ^ C _ rent 



ts 


22.10 


22.70 


22.40 




21.11 


22.48 


21.79 


h 


22.96 


22.89 


22.63 


col. 

Nothir 


21.50 22.46 

ig significant. 





1 



♦ See Tables 8-11, which this table supplements 
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APPENDIX G 



REPORT ON THE EFFECTIVENESS 
OF SPEECH INSTRUCTION 
IN THE RHETORIC PROGRAM 



o 



Report on the Effectiveness of Speech Instruction 
in the Rhetoric Program 
John Waite Bowers* 

University of Iowa 



Introduction . The Rhetoric Program at the University of Iowa 
combines instruction in writing, speaking, reading, and listening 
skills, emphasizing the commonality of rhetorical principit;, 2‘'.ross 
all modes of communication. In the typical Rhetoric class (4 Sih.), 
a student writes eleven themes, gives eight speeches, and fulfills 
about eight reading and listening assignments. These assignments 
and the accompanying instruction are divided about evenly between 
exposition and argument. 

In the fall of 1964, more than three hundred students were 
excuced from the Rhetoric requirement in an attempt to analyze 
differences in the writing, at various intervals, of students who 
do and students who do not take the course. The experiment reported 
here "hitch-hiked” on the writing experiment to determine the effect 
of speech instruction in Rhetoric. 

Subjects . During the fall semester, 40 subjects were randomly 
chosen from the no-Rhetoric pool of subjects (hereafter referred to 
as 10:6 subjects). These subjects were then matched on the bases 
of sex and ACT scores with 40 subjects taking 10:3 (accelerated 
Rhetoric) at that time (hereafter referred to as 10:3 subjects). 
Although every effort was made to test all subjects selected (see 
Addendum A), a few subjects were lost. Hie group finally tested 
included 38 10:3 subjects and 36 10:6 subjects. 

The rating instrument . Members of the Rhetoric faculty devel- 
oped a rating instrument consisting of 50 bi-polar, seven-level 
scales organized under the concepts "Content," "Language," and 
"Delivery," This instrument was pre-tested by having 11 raters use 
it on a film containing 5 student speeches. The resulting ratings 
were correlated across raters and across scales. On the basis of 
these correlations, 15 scales were discarded so that the final rating 
instrument (Addendum B) consisted of 35 scales organized under the 3 
concepts. These scales were further divided into nine relatively 
independent clusters which we labeled (1) analysis/supporting mate- 
rial, (2) perception of argument, (3) audience adaptation, (4) evi- 
dence documentation, (5) confidence, (6) fluency, (7) language 
appropriateness, (8) enthusiasm/activity, and (9) oral style. 

The raters , Raters in all phases of the experiment were Rhet- 
oric instructors hired through the Department of Speech and Dramatic 
Art who had at least one year*s experience teaching in the Program. 

*Professor Sam Becker provided extensive and valuable help in the 
development and interpretation of statistical data on the rating instru- 
ment. 
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On ths basis of the pre-test of the rating instrument, the 3 least 
reliable raters were dropped from the study. In the testing session, 
each speech was rated by two raters paired on the basis of high inter- 
rater agreement in the pre-test of the rating instrument. Raters had 
no way of knowing which subjects were 10:3 and which were 10:6. All 
raters were paid for their time by the College of Liberal Arts. 

The test . The test taken by the students at the end of the se- 
me steFn[A3c[endum C) was of the form typically used in practical speech 
tests in the Program. Subjects were given a list of general topics 
and instructed to prepare a four- to five-minute argumentative speech 
on one of them in a fifty-minute period. They then were divided into 
groups of 10 and sent to classrooms where they delivered their speeches 
for their peers and the two raters. 

Results . A ^-test for the difference between two means showed 
that the 10:3 students performed significantly better than did the 10:6 
students (t»2.54, £<.01). The mean difference was 24.27, the range of 
scores 193. Mean scores of 10:3 students were also higher than mean 
scores of 10:6 students on each of the 9 independent categories (see 
section on The rating instrument) . The chi square statistic indicates 
£<.025 for this consistent superiority. 

Interpretation . Instruction in speech in the Rhetoric Program 
significantly affects student speech behavior in what the Rhetoric 
faculty considers desirable directions. 






er|c 
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ADDENDUM A 

THE UNIVERSITY OF IOWA 

IOWA CITY, IOWA 5224C 



The Bhetoric Troffam 







January 6, 1965 



Dear 



Whether or not you are continuing to have a part in the Rhetoric 
experiment, we ask that you take part in a brief, supplementary exper- 
iment this January. 

The conditions 6f the regular experiment affort an unusual oppor- 
tunity to test the effectiveness of the Rhetoric Program instruction 
in speech as well as in writing. As you can readily see, it is of 
considerable importance to know what kind of effect Rhetoric has on the 
speaking performance of the different kinds of students required to 
take the course. Consequently, we have selected you, along with other 
students from the experimental and control groups, to take a special 
speech examination at the time and place checked below: 

Friday, January 15, 6:30-8:30 p.ra. 121-A SH 
Friday, January 15, 7:30-9:30 p.m. 121-A SH 



Please write the time and place on your calendar now. It is necessary 
for you to appear promptly. This examination takes precedence over 
all other events. We hope that by scheduling tne exam early on ^Friday 
e^ning we have minimized any interference with your personal affairs. 

If it is important for you to take the exam at the time other than the 
one for which you have been scheduled, please telephone Professor John 
Bowers, Speech Supervisor of the Rhetoric Program, Ext. 2069, by Monday, 
January 11. 

The speech examination, like the regular Rhetoric final speech 
exam, will consist of two phases: (1) when you arrive at 121-A SH, 

you will be given a list of topics from which you will select one on 
which you will have fifty minutes to prepare a four-minute extempor- 
aneous speech, and (2) when you are notified by the proctor, you will 
go to another room where you will take turns with seven other students 
to deliver the speech for rating by several specially trained instructors. 

If you have been taking 10:3 this semester, this special speech 
examination will replace the regular Rhetoric final speech exam for you; 
and, in order to equalize your motivation with that of students in the 
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experiaental group, we will consider that you have exempted yourself 
from the speech aspect of Rhetoric if you take this special examina- 
tion aF'indicated, no matter what your rating is. However, we hope 
that you will live up to the spirit of the experiment and do your 
best, and we will inform you of your rating so that you will know how 
well you did. (If a 10:3 student selected for this special speech 
examination neglects to take it, he will receive an "Incomplete” for 
his final Rhetoric grade this semester.) 

If you are a 10:6 student, excused from the Rhetoric requirement 
in the first place, we hope that you will take this added examination 

in the spirit of the experiment, too, and do your best, even though 

the results will not affect your standing in any way. We will inform 
you of your rating, too, so that you will know how well you did even 

though you have not had Rhetoric instruction. (If any 10:6 student 

selected for this special speecht .examination neglects to take it, he 
will be dismissed from the experiment and be required to fulfill the 
Rhetoric requirement like any other freshman.) 

Again, thank you for helping us conduct this supplementary exper- 
iment. The results will be important not only for the University of 
Iowa but for colleges and universities the country over. 



Yours sincerely 




Richard Braddock 
coordinator. Rhetoric Program 






Dewey B. Stuit 
Dean 



ERIC 
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ADDENDUM B 
DELIVERY 



Enthusiastic 


• 

• 


• 

• 


• 

• 


X 


X 


1 


Face active 


: 


: 


• 

• 


: 


: 


: 


Voice active 


• 

• 


: 


• 

• 


• 

• 


• 

• 


• 

• 


Body active 


: 


• 

• 


• 

• 


• 

• 


• 

• 


: 


Voice emphatic 


• 

• 


• 

• 


: 


: 


: 


• 

• 


Manner 


: 


: 


: 


: 


: 


: 


appropriate 
to material 


Eyes .direct 


: 


• 

• 


: 


: 


: 


•i 


Body free 


• 

• 


: 


• 

• 


: 


: 


: 


Audience 


oriented 


1 


: 


: 


: 


: 


• 

• 


Personally 

involved 


: 


: 


• 

• 


• 

• 


: 


: 


Healthy 


tension 


: 


: 


• 

• 


: 


: 


: 


Poised 


: 


: 


• 

• 


: 


: 


: 








LANC5UAGE 








Appropriate 


to audience 


1 


: 




j 


• 

• 


• 

• 






Face passive 
Voice passive 
Body passive 
Voice patterned 



Manner 

inappropriate 
to material 



Eyes lectern- 
bound or evasive 



Body lectern-bound 



Non-adaptive 



Personally 

uninvolved 



Extreme tension 



Terrified 



Inappropriate 
to audience 
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2 



Fluent 


t : 


• 

• 


: 


: 


: 


• Painful 


Clever 


: : 


: 


: 


: 


: 


. Dull 


Original 


: : 


: 


: 


! 


! 


Hackneyed 


V, 

Easy to 
follow 


: : 


• 

• 


: 


! 


• 

• 


Hard to follow 


Conversa- 

tional 


: : 


• 

• 


: 


e 

• 


! 


Stilted 


Oral style 


: ! 


: 


: 


t 


! 


Written style 


Natural 

word 

order 


t ! 


• 

• 


! 


! 


• 

• 


Artificial 

word 

order 





GONTENT 



Central idea 
significant 


• 

• 


• 

• 


• 

• 


t 


• 

• 


• 

• 


Central idea 
trivial 


Analysis clear 


s 

a 


: 


2 


• 

• 


• 

• 


2 


Analysis confused 


Evidence 

focused 


t 


t 


t 


: 


: 


2 


Evidence 

diffuse 


Evidence 

real 


t 


: 


: 


2 


: 


2 


Evidence 

hypothetical 


Evidence 

specific 


: 


• 

• 


: 


: 


: 


2 


Evidence 



general 
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3 





f 

r 



& 






•tr 

I- 

f 

i 

i* 



K 












Analysis 

logical 


: 


• 

• 


: 


: 


: 


: 


Analysis 

illogical 


Evidence 

amplified 


: 


: 


: 


: 


: 


: 


Evidence 


Evidence 

documented 


• 

• 


• 

• 


• 


: 


: 


: 


skeletal 

Evidence 

undocumented 


Special 

knowledge 


• 

• 


• 

• 


• 

• 


: 


: 


: 


Common knowledge 


Propositions 

controversial 


t 


• 

• 


• 

• 


: 


: i- : 


Propositions 

truistic 


Content 

interesting 


: 


• 


t 


• 

• 


• • 

• • 


Content boring 




Adapts to 
audience 



beliefs 


• 


m 

• 


: 


: 


: 


• 

• 


Demonstrates 

central 

ideals 

iTiportance 


t 


: 


: 


i 


: 


: 


Evidence 

relevant 


: 


t 


! 


t 


: 


t 


Accounts 

for 

opposition 


t 


: 


: 


t 


t 


: 



Ignores 

audience 

beliefs 



•1 



Assumes 
central 
idea * s 
ingportance 



Evidence 

irrelevant 



Suppresses', distort* 
ignores 
opposition 



ADDENDUM C 



RHETORIC FROGRAM 

Speech Examination, January 15, 196^ 



Tout assignment is to present a four- to fiire-mlnute argumentative speech 
on one of the topics listed below. The room chairman will enforce the 
five-minute time limit strictly, so be sure to narrow the topic you 
choose to a controversial proposition which you can " 

that time. For exan^le, if you chose the topic "Capital Punlshmen , 
you might decide to speak on the proposition, "Capital punishment does 
not deter capital crimes.” 

You should impress your audience with specific, documented, supporting 
material and sound reasoning based upon your experience, general knowledge, 
and specific reading. You may use notes in any way you wish. In general, 
however, the raters will penalize you for excessive reading or word-for- 
word memorization of material. Be direct and conversational. 



1. Primaiy education 

2. Housing at the U of I 

3. Problems in labor 
It, State revenue 

5. Private schools and the taxpayer 

6 . Ci.vil rights organizations 

7. Right-wing strategies 

8 . Religion and Congress 

9. The U.S. and Europe 

10. The cost of dying 

11. Art and artists 

12. Marriage and divorce 

13 . Traffic and traffic laws 

lit. Freedom of speech for students 



